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16.  Abstract 

In  May  1980  the  Federal  Aviation  Administration  completed  construction  of  a  Radar 
Training  Facility  (RTF)  in  Oklahoma  City.  The  primary  objective  of  the  RTF  is  to 
closely  duplicate  the  specialized  operational  environment  existing  at  automated 
Terminal  and  En  Route  facilities  as  well  as  to  have  the  capability  of  synthesizing  a 
wide  variety  of  air  traffic  control  situations.  Corresponding  to  each  radar  training 
sector,  there  is  a  manual  controller  position,  a  ghost  position,  and  three  pilot 
positions.  The  ghost  positions  serve  as  the  adjacent  sectors  to  the  radar  position, 
while  persons  at  the  pilot  positions  control  the  flight  of  the  aircraft  simulated  at 
the  radar  positions. 

Errors  at  the  pilot  position  confound  the  scoring  procedure  for  evaluating  the 
performance  of  the  trainee  in  the  radar  position.  This  study  was  designed  to  deter¬ 
mine  the  expected  error  rates,  what  Inputs  result  In  the  most  errors,  how  long  It 
takes  to  train  pilots  to  proficiency,  and  who  should/could  operate  the  pilot  posi¬ 
tions.  With  respect  to  the  latter,  error  rates  and  training  time  were  compared  for 
three  groups;  namely,  air  traffic  control  specialist  trainees,  community  persons,  and 
handicapped  persons. 

The  results  of  the  study  Indicate  that  all  three  groups  can  learn  to  operate  the 
pilot  position  within  a  reasonable  length  of  time  with  an  acceptable  error  rate.  A 
recommendation  is  made  to  employ  handicapped  and  community  persons  at  the  pilot 
position  In  order  to  maintain  a  stable,  competent  group  of  RTF  pilots. 
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SYSTEM  PERFORMANCE,  ERROR  RATES,  AND  TRAINING  TIME  FOR  RECENT  FAA  ACADEMY 
NONRADAR  GRADUATES,  COMMUNITY  PERSONS,  AND  HANDICAPPED  PERSONS  ON  THE  RADAR 

TRAINING  FACILITY  PILOT  POSITION 


I.  Introduction. 

A.  Background.  The  original  simulators  used  In  air  traffic  control  ( ATC) 
training  were  "patches"  added  to  the  operational  field  systems  of  the  NAS-A 
and  ARTS-III.*  The  patches  permitted  flexible  training  at  designated  positions 
without  Interfering  significantly  with  the  operational  positions.  These 
prototype  simulators  resulted  In  at  least  two  major  notions  related  to  using 
simulation  for  radar  training.  First,  the  value  of  computer-driven  simulation 
for  training  purposes  was  firmly  established.  Second,  several  problems 
associated  with  using  operational  field  systems  in  a  training  mode  were 
identified.  The  Institute  for  Defense  Analysis  (IDA)  study  on  the  training  of 
air  traffic  controllers  discussed  some  of  these  problems  and  suggested  that  a 
standardized  computer-driven  program  should  be  established  by  the  Federal 
Aviation  Administration  (FAA)  to  provide  basic  radar  training.  The  IDA  study 
further  suggested  that  the  radar  training  should  be  pass/fail  to  Identify 
those  persons  who  did  not  demonstrate  the  potential  to  perform  proficiently  in 
a  radar  environment  (2). 

In  July  1976  engineering  requirements  were  completed  by  the  FAA  for  a  radar 
training  system  (1).  During  that  same  month,  the  FAA  Administrator  approved  the 
procurement  and  construction  of  the  Radar  Training  Facility  (RTF)  to  be 
located  at  the  FAA  Academy  in  Oklahoma  City,  Oklahoma. 

In  October  1977  the  FAA  completed  a  program  implementation  plan  that 
outlined  the  development  and  Implementation  of  the  RTF.  The  contract  for  the 
development  of  the  computer-driven  simulator  training  system  was  awarded  to 
Loglcon,  Tactical  and  Training  System  Division,  San  Diego,  California,  in 
January  1978.  Groundbreaking  for  the  construction  of  the  new  RTF  at  the  FAA 
Academy  was  held  on  December  22,  1977. 

B.  RTF  Training  System  and  Laboratory  Configuration.  The  primary  objec¬ 
tive  of  the  RTF,  as  stated  In  the  engineering  requirements,  is  to  closely 
duplicate  the  specialized  operational  environment  existing  at  automated 
Terminal  and  En  Route  facilities  as  well  as  have  the  capability  of  synthe¬ 
sizing  a  wide  variety  of  air  traffic  control  situations.  These  situations 
would  be  based  on  a  reference  data  base  created  through  scenario  programs  with 
a  full  range  of  control  necessary  to  establish  a  realistic  simulation  of 
actual  aircraft  traffic  under  a  variety  of  conditions. 

To  accomplish  this  objective,  Loglcon  proposed  that  four  Independent 
laboratories  be  constructed.  Figure  1  describes  how  the  laboratories  are 
configured. 

♦Computer  systems  employed  in  En  Route  and  Terminal  air  traffic  control, 
respectively. 
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P  -  Pilot  Console 
6  -  Ghost  Console 
R  -  Radar  Training  Console 
M  -  Manual  Controller  Trainee  Position 
En  Route  "D"  Controller  or 
Termlnol  "HO*  Controller 


MI  -  Master  Instructor  Position 
PS  -  Pilot  Supervisor  Station 
I  -  Instructor  Station 

FSP  -  Flight  Strip  Printer 


FIGURE  T.  Typical  laboratory 

Corresponding  to  each  radar  training  sector,  there  is  a  manual  controller 
position,  a  ghost  position,  and  three  pilot  positions.  The  ghost  positions 
will  serve  as  the  adjacent  sectors  to  the  radar  positions  or  other  facilities, 
while  the  pilot  positions  will  control  the  flight  of  the  aircraft  simulated  at 
the  radar  position.  Each  of  the  four  laboratories,  two  En  Route  and  two 
Terminal,  can  train  six  radar  control  students  simultaneously.  There  Is  an 
Instructor  station  at  each  of  the  radar  positions  and  a  master  Instructor 
station  and  system  monitor  station  In  each  of  the  four  laboratories.  Table  1 
summarizes  the  totals  of  the  various  positions  In  the  RTF. 

Table  1.  RTF  Positions 

24  Radar  Positions 

24  Data  Handler  Positions 

72  Pilot  Positions 

24  Ghost  Positions 

4  Master  Instructor  Stations 

4  System  Monitor  Stations 

24  Instructor  Stations 
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Figure  2  describes  the  system  configuration  for  operating  the  positions 
and  stations  In  each  laboratory.  The  training  sectors  are  controlled  by  a 
Digital  Equipment  Corporation  (DEC)  PDP  11/60  computer  with  a  PDP  11/34 
computer  serving  as  an  interface  between  the  PDP  11/60  and  the  operating 
positions. 


SYSTEM  MONITOR 
CONSOLE 


P0P-II/34  PROCESSOR 
•  2  OR  3  PER  LABORATORY 


SECTOR  STUOENT  CONSOLE 
•  6  POSITION  PER 
LABORATORY 


PLIGHT  STRIP 
PRINTER 

•  3  PLIGHT  STRIP 
PRINTERS  ( I  PER 
STUOENT  PAIR) 


IB  PILOT  a  B  GHOST  POSITIONS 
PER  LABORATORY 
•  3  PILOT  B  I  GHOST 
POSITIONS  POR  EACH 
STUOENT  CONSOLE 


FIGURE  2.  System  configuration. 

The  training  process  involves  three  sequential  systems  of  operation: 
SCENARIO  GENERATION  —  «*•  REAL-TIME  TRAINING  — ►  PERFORMANCE  MEASUREMENT. 

Scenario  generation,  illustrated  in  Figure  3,  is  the  non-real-time 
process  of  building  exercises  and  evaluation  problems  for  the  system. 

Aircraft  characteristics,  flight  plans,  and  other  essential  information  of 
this  type  are  stored  in  the  Universal  Data  Files  (UDF).  The  exercise  Is  built 
by  first  selectively  retrieving  intermediate  files  and  then  creating  other 
Intermediate  data  files  from  the  universal  data  base  through  the  scenario 
management  program. 

The  real-time  component.  Illustrated  in  Figure  4,  utilizes  the  scenario 
management  files  to  generate  the  actual  radar  simulation  exercise.  The 
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FIGURE  4.  Real-time  training. 
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real-time  component  drives  the  display  at  the  radar  position.  Aircraft  move¬ 
ment  is  controlled  through  the  pilot  and  ghost  positions  according  to  the 
instructions  those  positions  receive  from  the  controller  trainee  or  in  some 
cases  forms  a  scenario  prompt  that  appears  on  the  pilot  or  ghost  cathode-ray 
tube  (CRT)  at  those  responsive  positions.  All  actions  taken  during  the 
operation  of  the  real-time  training  exercise  are  recorded. 

At  completion  of  the  exercise,  the  computer  will  analyze  the  recorded 
actions  to  determine  violations  of  separation  standards  and  other  pertinent 
information,  such  as  delay  times,  in  order  to  evaluate  the  student's  ability 
to  move  air  traffic  "safely  and  expeditiously."  The  process  of  student 
performance  measurement  is  illustrated  in  Figure  5. 


FIGURE  5.  Student  performance  measurement. 


C.  Purpose  of  This  Study.  This  study  is  concerned  with  the  operation 
of  the  pilot  position  in  the  real-time  training  program.  The  pilot  position 
console,  a  modified  version  of  the  Hazeltine  Modular  One  Terminal,  consists  of 
a  tabular  CRT  display  with  associated  controls  and  data  entry  devices  (see 
Figure  6). 


Three  pilot  positions  support  each  radar  position.  The  pilot  positions 
are  used  to  maneuver  the  aircraft  targets  displayed  on  the  radar  CRT.  The 
data  entry  and  readout  capabilities  of  the  pilot  position  permit  rapid 
entries  and  changes  to  the  aircraft  flight  status  according  to  the  Instructions 
received  from  the  radar  air  traffic  control  trainee. 


It  is  apparent  from  Figure  4,  which  describes  the  real-time  training 
system,  that  the  accuracy  of  the  Inputs  (responses)  from  the  pilot  position 
affect  the  performance  score  of  the  trainee  who  is  in  the  radar  position. 
Suppose,  for  example,  that  an  aircraft  is  Instructed  by  the  trainee  controller 
to  turn  to  a  60°  heading  but  the  pilot  inadvertently  turns  to  a  260°  heading. 
An  error  such  as  this  would  certainly  confound  the  scoring  procedure  for 
evaluating  the  performance  of  the  trainee  in  the  radar  position.  The  first 


questions  that  arose  in  this  study  were  related  to  the  expected  accuracy  of 
the  inputs  from  the  pilot  position:  What  are  the  expected  error  rates?  What 
inputs  will  result  in  the  most  errors?  What  inputs  should  be  emphasized  more 
in  pilot  training?  Should  the  keyboard  arrangement  be  modified  to  aid  in 
accuracy? 

Another  unknown  parameter  was  training  time.  The  schedule  of  laboratory 
use  must  include  an  instructional  component  for  the  pilot  position.  The 
question  of  how  long  it  takes  to  train  to  proficiency  needed  to  be  answered. 

A  third  important  topic  of  inquiry  in  the  study  was  the  feasibility  of 
using  the  RTF  computer  facilities  for  computer-assisted  programed  Instruction 
for  pilot  training.  This  required  the  creation  and  evaluation  of  a  prototype 
training  system,  and  an  evaluation  of  the  feasibility  of  modifying  the  RTF 
training  equipment  for  use  in  pilot  training. 


FIGURE  6.  Pilot  position. 
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The  last  focus  of  the  study  was  on  who  could/should  operate  the  pilot 
position.  If  ATC  trainees  were  to  operate  the  pilot  positions,  they  would 
spend  three  times  as  much  time  on  the  pilot  positions  as  they  would  spend 
being  trained  on  the  radar  position.  This  is  undesirable  since  the  pilot 
position  will  not  be  used  anywhere  in  the  controller's  work  after  training. 

In  other  training  systems  similar  to  the  RTF,  pilot  operators  are  hired  to 
operate  the  pilot  position.  Since  operating  the  pilot  position  is  a 
sedentary  task,  it  could  be  performed  by  many  handicapped  persons,  such  as 
those  confined  to  wheelchairs.  One  of  the  purposes  of  the  study  was  to 
compare  error  rates  and  training  time  for  ATC  trainees,  people  from  the 
general  community,  and  handicapped  people. 

II .  Methods. 

A.  Subjects.  There  were  three  separate  groups  of  subjects.  Table  2 
gives  a  description  of  the  three  groups  by  number,  sex,  and  age. 

Those  in  Group  I  were  recent  graduates  of  the  FAA  Academy  in  nonradar  ATC. 
The  selection  of  Group  I  subjects  was  based  on  Academy  scores  (Table  3), 
minority  status,  sex,  and  availability  for  the  study. 

Group  II  consisted  of  paid  subjects  from  the  Greater  Oklahoma  City 
community  who  were  selected  on  the  basis  of  the  following  criteria:  They  were 
required  to  be  between  the  ages  of  18  and  45,  nonhandicapped,  nonstudent,  and 
of  average  or  better  intelligence  (not  classified  as  mentally  handicapped). 

Group  III  consisted  of  paid  subjects  from  the  Greater  Oklahoma  City 
community  who  met  the  following  criteria:  They  were  required  to  be  between 
the  ages  of  18  and  50,  handicapped  (but  with  normal  vision  in  at  least  one 
eye,  normal  hearing  in  at  least  one  ear,  and  at  least  one  arm  and  hand  with 
no  disability  exceeding  25  percent),  and  of  average  or  better  intelligence 
(not  classified  as  mentally  handicapped).  Table  4  describes  Group  III  by 
handicap. 

Two  concerns  expressed  by  Academy  personnel  regarding  pilot  training  were 
included  in  the  study  design:  (1)  Does  note  taking  during  training  impede 
or  enhance  the  learning  process?  and,  (2)  How  much  academic  training  is 
required?  To  answer  these  questions,  each  group  was  randomly  divided  into  two 
equal  sections;  one  section  was  allowed  to  take  notes  during  the  instructional 
phase  and  the  practice  and  evaluation  exercises  while  the  other  section  was 
not  allowed  to  take  notes  during  these  times  and  had  to  rely  on  memory. 

Groups  II  and  III  were  randomly  subdivided  further  into  two  equal  sections. 
Section  I  received  approximately  4  hours  of  classroom  academic  instruction 
and  Section  II  received  approximately  8  hours. 

B.  Training  System.  The  complete  training  system  consisted  of  two  inde¬ 
pendent  components,  a  computer  system  and  a  communication  system.  The 
computer  system  comprised  a  DEC  11/34  computer,  dual  floppy  disk  drives,  an 
LA36  Decwriter,  and  12  modified  Hazeltine  Modular  One  terminals.  Each 
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Table  2.  Description  of  Subjects  by  Sex  and  Age 


Group 

Subject 

Number 

Men 

Sex 

Women 

Age 

Group  I 

SI 

X 

21 

S2 

X 

24 

S3 

X 

26 

S4 

X 

25 

S5 

X 

24 

S6 

X 

29 

S7 

X 

29 

S8 

X 

30 

S9 

X 

20 

S10 

X 

27 

Sll 

X 

24 

Group  II 

S12 

X 

33 

SI  3 

X 

S14 

X 

SI  5 

X 

26 

S16 

X 

27 

SI  7 

X 

S18 

X 

38 

SI  9 

X 

21 

S20 

X 

37 

S21 

X 

32 

S22 

X 

S23 

X 

Group  III 

S24 

X 

47 

S25 

X 

S26 

X 

S27 

X 

S28 

X 

S29 

X 

26 

S30 

X 

25 

S3! 

X 

39 

S32 

X 

29 

S33 

X 

26 

S34 

X 

43 

TOTALS 

N=34 

N=1 7 

N=1 7 
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Table  3.  Distribution  of  Academy  Graduates'  Composite  Scores 

(N=ll) 


Score 


70  2 

71  1 

72  1 

73  2 

74  1 

75  0 

76  0 

77  0 

78  2 

79  1 

80  0 

81  1 


Table  4.  Description  of  Handicapped  Group  by  Handicap 

Type  of  Handicap 


Subject 

Number 


Vi sual 


Wheelchair  or 
Immobile 


Hand  or  Arm 
Disability 


Miscellaneous 


524 

525 

526 

527 

528 

529 

530 

531  X 

532 

533 

534 


X 

X  X 

X 

X 

X 

X 

X  X 


X 

X 

X 

X 


X 

X 


terminal  had  both  a  keyboard  and  a  CRT.  The  communication  system  consisted  of 
a  Superscope  Stereo  cassette  recorder  (GS200)  and  14  standard  ATC  headsets, 
one  attached  tc  each  position  and  two  available  for  system  monitoring.  Figure 
7  is  a  graphic  representation  of  both  the  system  equipment  and  the  individual 
position  equipment. 
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1 

RECORDING 

STATION 

POP  11/34 
COMPUTER 

PRINTER 

TERMINAL 


PENCIL  AND  PAPER 
NOTE  TAKING  GROUP 


HEADPHONE  -i 
JACKS  ^ 


H 


MEMORY 

GROUP 


■CT 


IT 


FIGURE  7.  Equipment  configuration. 

The  command  messages,  both  for  the  evaluation  and  for  the  practice 
exercises,  were  first  recorded  on  floppy  disks  by  using  the  computer  input 
mode.  After  the  messages  were  recorded  on  floppies,  they  were  played  back  and 
audio  recordings  of  the  messages  were  made  on  cassette  tapes.  The  same  timing 
was  used  for  both.  During  training  and  evaluation  exercises,  the  prerecorded 
messages  were  broadcast  simultaneously  through  the  headset  to  each  pilot 
position.  Each  command  was  broadcast  only  one  time.  The  keyboard  response  to 
each  command  by  each  subject  was  recorded  by  the  computer  as  soon  as  the 
subject  depressed  the  ENTER  key  on  the  keyboard.  The  computer  then  compared 
the  entered  response  with  the  correct  message  which  had  been  previously 
recorded  on  the  floppy  disk.  If  the  response  was  correct,  the  computer  erased 
the  subject's  response  from  the  CRT.  If  the  response  was  incorrect,  the 
computer  recorded  the  error  on  another  floppy  disk  and  also  transmitted  an 
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error  message  to  the  subject's  CRT.  Errors  were  recorded  under  five  different 
categories.  These  are  listed  and  described  in  Table  5.  The  subject  could 


Table  5.  Summary  of  Error  Definitions 


Error 

Number 

Error  Type 

Definition 

1 

Flight  ID  Error 

A  correct  command  was  entered  but  the  wrong  flight 
(or  no  flight)  was  selected  at  the  time  of 
command  message  entry. 

2 

Format  Error 

The  correct  flight  was  selected  at  entry  time  but 
an  incorrect  format  was  detected  in  the  syntax 
rules. 

3 

Flight  ID  and 
Format  Error 

The  wrong  flight  (or  no  flight)  was  selected  and 
an  incorrect  format  was  detected  in  the  syntax 
rules. 

4 

Content  Error 

A  correct  flight  was  selected  and  no  syntax  errors 
were  detected  but  the  content  of  the  command  was 
in  error. 

5 

Flight  ID  and 
Content  Error 

The  wrong  flight  (or  no  flight)  was  selected  and 
the  content  of  the  command  was  in  error. 

then  either  modify  the  message  or  delete  the  incorrect  message  and  reenter  the 
message  correctly.  When  the  ENTER  key  was  again  depressed  after  the  correc¬ 
tion,  the  message  was  again  cycled  through  the  computer  to  determine  if  it 
was  correct.  This  process  was  repeated  with  each  command. 

The  functionally  oriented  keyboard  was  divided  into  software-related 
functional  areas  that  were  color  coded.  Figure  8  shows  the  keyboard  and  the 
color-coded  areas.  The  first  area,  consisting  of  orange  keys,  was  the  flight 
identifier  area.  These  12  keys  were  line  numbers  that  corresponded  to 
aircraft  listed  on  the  CRT  tabular  display  at  each  position.  They  were  used 
as  a  fast-entry  reference  to  control  the  flight  status  of  the  aircraft  targets 
(see  Figure  8).  Table  6  contains  the  aircraft  identifications  and  the 
corresponding  line  numbers.  One  key  was  selected  for  each  command  given;  for 
example,  the  orange  key  marked  "Line  2"  was  depressed  whenever  the  command 
referred  to  "American  fourteen  twenty."  (See  Table  6.)  The  second  area, 
consisting  of  16  blue  keys  across  the  top  center  of  the  keyboard,  was  called 
the  function  key  area  (see  Figure  8).  These  keys  initiated  the  primary 
function  of  each  message  and  alerted  the  computer  that  subsequent  Information 
to  Initiate  or  change  an  aircraft's  status  would  be  forthcoming.  For 
example,  "cleared  as  filed"  was  input  by  depressing  the  DEPART  function  key. 
Some  input  messages  required  more  than  one  function  key  (see  Appendix  1  for  a 
listing  of  the  multifunction  messages  employed  in  the  study).  The  next  area 

11 


Table  6.  Predefined  Flight  Identifiers 


Line 

Number 

Flight  ID 

Flight  Phraseology 

1 

UA365 

United  Three  Sixty  Five 

2 

AA1420 

American  Fourteen  Twenty 

3 

R1 8745 

Army  18745 

4 

N7398B 

November  73  Niner  8  Bravo 

5 

NA217 

National  Two  Seventeen 

6 

VV7K512 

Navy  7  Kilo  512 

7 

N71Z 

November  71  Zulu 

8 

A28160 

Air  Force  28160 

9 

BN78 

Branlff  Seventy  Eight 

A 

VM72240 

Marine  72240 

B 

N684WX 

November  684  Whiskey  X-Ray 

C 

DL403 

Delta  Four  Oh  Three 

Note:  There  are  five  airline,  four  military,  and 
three  general  flights  identified  above. 


ON  LIGHT 


6 

CXL 

FLT 

FR7 

LINE 

LINE 

1 

2 

LINE 

LINE 

3 

4 

LINE 

LINE 

a 

6 

LINE 

LINE 
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LINE 

LINE 
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A 
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LINE 
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C 

HEADING 

ALT 

SPEED 

ROUTE 

CLRNC 

LIMIT 

HAND 

OFF 
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REPORT 
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FIGURE  8.  Pilot/ghost  position  keyboard. 


of  keys  (subcommand  function  keys)  was  located  in  the  center  left  of  the 
keyboard  or  the  action  key  area.  The  action  keys  were  dark  grey  with  the 
exception  of  one  action  key  (RESUME)  located  in  the  green  key  area.  The 
action  keys  gave  the  operation  or  action  to  be  performed  for  the  specified 
function.  For  example,  "turn  left"  was  Input  using  the  HEADING  function  key 
and  the  left  arrow  action  key.  The  light-grey  or  alpha  keys  were  used  to  Input 
any  alpha  characters  required  In  the  messages,  such  as  a  STAR  (Standard 
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Arrival)  or  SID  (Standard  Instrument  Departure)  name.  The  next  area,  the 
black  keys  at  the  bottom  of  the  alpha  area,  consisted  of  seven  keys  that  were 
mainly  used  to  move  the  cursor  on  the  CRT  and  to  correct  or  edit  Input  errors. 
These  keys  were  called  either  entry  area  keys  or  edit  keys.  The  yellow  keys 
located  at  the  far  right  of  the  keyboard  were  called  numeric/direction  keys. 
The  keys  were  depressed  to  represent  numbers  In  the  messages,  such  as  an 
altitude  or  heading.  If  the  direction  was  required  In  the  Input  message,  the 
DIR  key  In  the  action  area  was  first  depressed,  then  the  corresponding  yellow 
key  was  used  to  Indicate  the  compass  direction.  For  example,  the  yellow  key 
marked  "7"  and  HSWH  Indicated  the  direction  southwest  when  the  DIR  key  was 
depressed  Immediately  prior  to  It.  The  red  keys  In  the  upper  rlghthand 
corner  of  the  keyboard  were  not  operative  except  for  the  DEL  LINE  key,  which 
deleted  the  entire  entry  except  for  the  flight  ID  number.  The  CLEAR  DISPLAY 
key  In  the  green  area  was  depressed  when  the  subject  wished  to  delete  all  of 
his/her  entry  Including  the  flight  ID  number. 

Figure  9  illustrates  the  operational  steps  involved  In  the  training  system. 


FIGURE  9.  System  process. 


C.  Instructional  Design.  The  possible  Input  messages  for  the  pilot 
position  were  listed  by  the  manufacturer  In  an  appendix  to  the  computer 
program  functional  specifications.  Four  expert  judges,  all  air  traffic 
control  specialists,  served  as  a  panel  to  Identify  the  Input  messages  on  the 
list  that  are  most  frequently  used  In  air  traffic  control  simulation  problems 
and  the  messages  that  are  rarely  used.  The  Input  messages  rarely  used  were 
eliminated  and  a  new  list  containing  50  Items  was  formed.  The  new  list  was 
divided  Into  categories  of  messages  according  to  the  number  of  key  depressions 
required  to  Input  the  message.  Ten  categories,  placed  In  order  of  Increasing 
difficulty,  constituted  the  final  list  of  Input  message  types  taught  In  the 
pilot  training.  Examples  of  Input  messages  based  on  the  50  message  types  are 
listed  by  category  in  Appendix  1. 

The  amount  of  time  allowed  to  Input  a  particular  message  type  was 
established  as  follows.  Several  naive  noncontroller  subjects  were  used  to 
determine  how  much  time  a  beginner  would  require  to  input  each  message.  The 
average  time  required  by  the  naive  subjects  to  Input  the  messages  in  each  of 
the  categories  was  used  as  the  100-percent  time  allotment.  Table  7  contains 
the  100-percent  time  allotment  assigned  to  each  Input  message  category.  Each 
Input  message  category  was  taught  separately,  beginning  with  category  1. 
Instructors  taught  the  first  command  In  the  category  and  the  keys  necessary  to 
enter  the  command  at  the  keyboard.  Time  was  allowed  for  any  questions  or 
clarification,  then  subjects  were  given  practice  time  to  enter  the  command  at 
the  keyboard  In  the  form  of  a  dry  run.  The  dry  runs  consisted  of  3  to  5 
minutes  of  recorded  messages  of  only  that  command.  Time  was  then  given  for 
any  questions  before  going  on  to  the  next  command.  After  all  commands  in  a 
category  were  taught,  the  subjects  were  given  a  practice  problem  covering  all 
the  commands  in  that  category.  The  100-percent  time  limit  was  allowed  on  the 
first  run.  Two  practice  problems  A  and  8  for  each  category  were  designed  at 
the  100-percent  time  limit  and  the  same  two  problems  were  constructed  at  75 
percent  of  the  100-percent  time  limit.  The  practice  problems  were  alternated 
when  given  until  the  established  criterion  was  met  or  three  trials  were 
attempted.  The  criterion  to  be  reached  was  for  90  percent  of  the  subjects  to 
enter  90  percent  of  the  messages  correctly  (90  percent/90  percent).  If  the 
criterion  was  not  met  for  the  practice  problem,  there  was  a  remedial  review; 
then  the  second  problem  of  the  same  difficulty  was  administered  with  the  same 
time  limit.  If  the  criterion  again  was  not  met,  more  review  and  practice  time 
were  given,  and  the  practice  problem  was  repeated  to  a  maximum  of  three  trials 
before  proceeding  to  the  next  category.  An  Illustration  of  the  Instructional 
modules  Is  given  In  Figure  10. 

After  completing  the  10  categories,  the  subjects  were  given  five  evalua¬ 
tion  problems.  The  evaluation  problems  contained  50  messages  taken  from  each 
of  the  10  categories.  All  command  types  were  represented  In  the  problems. 

The  first  evaluation  problem  was  designed  at  100-percent  time  allotment.  The 
time  allotments  for  the  second,  third,  fourth,  and  fifth  evaluation  problems 
were  decremented  by  category  and  administered  as  shown  In  Figure  11. 
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FIGURE  10.  Practice  session  module. 
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FIGURE  11.  Evaluation  process  module. 


Subjects  were  given  three  attempts  to  reach  the  90-percent/90-percent 
criterion  on  the  evaluation  problems.  If  the  criterion  was  not  met  on  the 
first  trial,  there  was  a  remedial  review  and  practice  time  on  the  most 
frequently  missed  messages.  The  evaluation  problem  was  run  again.  Once  the 
criterion  was  met  or  the  three  attempts  were  made,  the  next  evaluation 
problem  was  run  with  the  input  time  allowed  decreased  as  indicated  on  Figure 
11.  The  usual  day  consisted  of  6  hours  of  training  with  a  1-hour  lunch  break 
and  a  10-minute  break  each  hour. 

D.  Measures. 

1.  When  a  comparison  took  place  within  the  computer  between  the 
messages  Input  by  the  subjects  and  the  correct  Inputs,  three  measures  were 
made : 

a.  Attempts:  Number  of  attempts  to  input  messages. 

b.  Errors:  Number  of  incorrect  entries. 

c.  Raw  Scores:  Computed  by  subtracting  errors  from  attempts. 

2.  Academy  scores  were  maintained  for  Group  I. 
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3.  The  Dial  Reading  Test  and  the  Directional  Headings  Test  were 
administered  to  all  subjects  prior  to  beginning  pilot  training. 

4.  Demographic  data  were  collected  by  means  of  a  biographical 
questionnaire  (see  Appendix  2). 

5.  Following  the  completion  for  instruction  and  practice  problems  for 
each  category,  an  evaluation  form  was  administered  (see  Appendix  3). 

6.  Following  the  completion  of  the  evaluation  problems,  a  keyboard 
evaluation  questionnaire  was  administered  (see  Appendix  4). 

7.  Lastly,  a  record  of  the  number  of  trials  required  to  complete  each 
category  was  recorded. 

E.  Analyses. 

1.  Descriptive  statistics  were  computed  for  all  relevant  variables  in 
the  form  of  means,  standard  deviations,  sample  sizes,  and  intercorrelations. 

2.  Scores  on  the  Dial  Reading  Test  and  the  Directional  Headings  Test 
were  regressed  on  raw  score  performance  in  the  evaluation  problems  to  deter¬ 
mine  how  well  the  test  predicted  performance  and  to  determine  the  appropriate 
weights  to  be  employed  in  forming  composites  for  selection  purposes.  A  cut 
point  was  established  by  forming  a  distribution  of  the  composite  score  (in 
rank  order)  and  raw  performance  scores.  A  judgment  was  then  made  by  observing 
where  the  raw  scores  made  a  drop  at  the  lower  end  of  the  composite 
distribution. 

3.  Data  on  the  evaluation  forms  were  summarized  by  totaling  the 
frequency  of  responses  for  each  item  on  each  response  scale. 

4.  A  three-way  multi  factor  analysis  of  variance  (Anova)  was  computed 
on  (i)  the  Independent  dimensions  of  group  membership,  (ii)  whether  or  not  the 
subjects  were  allowed  to  take  notes,  and  (ill)  whether  the  subjects  received 
"long"  academic  training  or  "short"  academic  training.  (See  Table  8  below  for 
the  orthogonal  design  matrix  employed  in  the  regression  to  perform  the 
Anovas.)  Raw  performance  score  was  used  as  the  dependent  variable. 

5.  Graphs  were  formed  by  plotting  attempts,  errors,  and  raw  scores  by 
command  categories  1  through  10  and  by  evaluation  exercises  1  through  5  for 
Groups  I,  II,  and  III.  The  number  of  trials  required  for  reaching  the 
criteria  were  plotted  across  exercises  1  through  5. 

6.  Amount  of  training  time  required  was  computed  by  summing  the 
number  of  hours  required  to  reach  proficiency. 
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Table  8.  Orthogonal  Design  Matrix 


_ Main  Effects _  _ Interactions 

Notes  vs.  Long/Short  Group 


No  Notes 
(N) 

Academic 

(A) 

Membershi p 
_ (6) _ 

N/A 

N/G 

A/G 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

-1 

1 

1 

-1 

1 

1 

-1 

-1 

1 

1 

-1 

1 

1 

-1 

1 

-1 

1 

1 

-1 

-1 

1 

-1 

-1 

1 

1 

-1 

1 

1 

0 

-2 

1 

0 

-2 

0 

-2 

1 

-1 

0 

-2 

-1 

0 

-2 

0 

2 

-1 

1 

1 

1 

-1 

-1 

-1 

1 

1 

-1 

-1 

1 

1 

1 

-1 

-1 

-1 

-1 

-1 

1 

-1 

1 

-1 

1 

-1 

-1 

1 

-1 

-1 

-1 

1 

1 

1 

-1 

1 

-1 

-1 

1 

0 

-2 

-1 

0 

2 

0 

-2 

-1 

-1 

0 

-2 

1 

0 

2 

0 

2 

III.  Results. 

Descriptive  statistics  on  all  relevant  variables  are  listed  in  Tables  9 
and  10.  The  primary  descriptive  statistics  to  note  are  the  intercorrelations 
between  raw  scores,  attempts,  and  errors  with  the  Dial  Reading  Test,  the 
Directional  Headings  Test,  Group  Membership,  note  taking  vs.  no  note  taking, 
and  long  or  short  academic  training. 

Table  11  lists  the  regression  model  for  the  Dial  Reading  Test  and  the 
Directional  Headings  Test  in  predicting  raw  score  performance  and  the  F  test 
for  significance.  Note  the  size  of  the  "p"  value  on  the  F  test. 

Tables  12  and  13  summarize  the  response  frequencies  on  the  practice 
problem  evaluation  form  and  the  keyboard  evaluation  form,  respectively. 

Table  14  is  the  source  table  for  the  multifactor  analysis  of  variance. 

Figures  12  through  20  contain  the  graphs  for  attempts,  raw  scores,  and 
errors  across  command  categories  and  evaluation  exercises  and  the  number  of 
trials  for  each  evaluation  exercise  for  all  three  groups. 

IV.  Discussion. 

A.  Descriptive  Statistics.  It  can  be  noted  in  Table  10  that  raw  scores 
correlate  0.674  with  the  Dial  Reading  Test  and  0.638  with  the  Directional 
Headings  Test.  This  indicates  high  positive  ability  for  these  two  tests  to 
predict  raw  score  performance.  Further,  raw  score  has  a  -0.575  correlation 
with  group  membership  indicating  a  high  relationship  between  being  in  a 


Table  9.  Means,  Standard  Deviations,  and  Sample  Sizes 
for  Pilot  Study  Variables 


_ Variables _ 

Attempts 
Error  Prob  1 
Error  Prob  2 
Error  Prob  3 
Error  Prob  4 
Error  Prob  5 
Error  Total 
Raw  Score 
Trials 

Error  Message  Type  1 
Error  Message  Type  2 
Error  Message  Type  3 
Error  Message  Type  4 
Error  Message  Type  5 
Error  Message  Type  6 
Error  Message  Type  7 
Error  Message  Type  8 
Error  Message  Type  9 
Error  Message  Type  10 
Dial  Reading 
Table  Reading  1 
Table  Reading  2 
Table  Reading  Total 
Directional  Heading  1 
Directional  Heading  2 
Directional  Heading  Total 
Group  Membership 
Paper-Pencil  Note  Taking 
Short/ Long  Academics 


Standard 

Sample 

Means 

Deviations 

Sizes 

46.63 

3.37 

28 

0.25 

0.52 

28 

1 .18 

1  .25 

28 

0.37 

0.74 

28 

1.61 

1.34 

28 

0.04 

0.19 

28 

3.04 

2.33 

28 

43.64 

4.18 

28 

2.61 

0.50 

28 

0.04 

0.19 

28 

0.43 

0.69 

28 

0.68 

0.94 

28 

0.39 

0.69 

28 

0.14 

0.36 

28 

0.21 

0.42 

28 

0.18 

0.48 

28 

0.14 

0.36 

28 

0.71 

0.90 

28 

0.11 

0.31 

28 

31 .71 

12.33 

31 

27.00 

10.15 

20 

26.30 

11.42 

20 

80.45 

28.91 

20 

19.91 

11.12 

34 

19.82 

11.61 

34 

39.74 

22.04 

34 

2.00 

0.82 

34 

1.50 

0.51 

34 

1 .48 

0.51 

23 

20 
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Table  11.  Regression  Model 


Model  1  Criterion  =  2 

Predictors  =4-  4  10-10 


R  =  0.6921 

RSQ  =  0.4789 

2  Iterations 

V 

BETA 

B 

4 

0.4389 

0.1488 

10 

0.2889 

0.0548 

REG.  CONST.  =  36.7446 


F-TEST  1  DRT  &  DHT  PREDICTION  OF  RAW  SCORE 

RSQ  FULL  =  0.4789  MODEL  1 

RSQ  REDUCED  =  0.0000  MODEL  0 

DIFFERENCE  =  0.4789 

DFN  =  2.  DFD  =  25.  F-RATIO  =  11.490  P  =  0.00049 


*  *  * 


i  Table  12.  Evaluation  Form  for  Practice  Modules 

!  Directions:  Circle  the  response  that  best  expresses  your  observation,  make 

comments,  where  appropriate. 

1.  Were  the  commands  taught  clearly? 

Average  Very 


Unclear 

Clear 

Clear 

CAT. 

1 

0 

CAT. 

1 

_ 

7 

CAT. 

1  - 

24 

CAT. 

2 

- 

0 

CAT. 

2 

- 

21 

CAT. 

2  - 

9 

1 

CAT. 

3 

- 

0 

CAT. 

3 

- 

9 

CAT. 

3  - 

23 

CAT. 

4 

- 

0 

CAT. 

4 

- 

13 

CAT. 

4  - 

18 

j 

[ 

CAT. 

5 

- 

0 

CAT. 

5 

- 

14 

CAT. 

5  - 

16 

9 

CAT. 

6 

- 

0 

CAT. 

6 

- 

10 

CAT. 

6  - 

20 

) 

CAT. 

7 

- 

0 

CAT. 

7 

- 

11 

CAT. 

7  - 

19 

1 

i  ' 

CAT. 

8 

- 

0 

CAT. 

8 

- 

8 

CAT. 

8  - 

22 

CAT. 

9 

- 

2 

CAT. 

9 

- 

10 

CAT. 

9  - 

17 

CAT. 

10 

- 

0 

CAT. 

10 

- 

7 

CAT. 

10  - 

22 

2 

no 

190 

i 
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Table  12  (continued)— 

2.  Was  enough  practice  time  given?  (Dry  runs) 


Not  Enough  Enough  Too  Much 

Time  Given  Time  Given  Time  Given 


CAT. 

1  - 

0 

CAT. 

CAT. 

2  - 

0 

CAT. 

CAT. 

3  - 

0 

CAT. 

CAT. 

4  - 

0 

CAT. 

CAT. 

5  - 

0 

CAT. 

CAT. 

6  - 

0 

CAT. 

CAT. 

7  - 

0 

CAT. 

CAT. 

8  - 

0 

CAT. 

CAT. 

9  - 

1 

CAT. 

CAT. 

10  - 

1 

CAT. 

2 


1 

-  21 

CAT. 

1 

9 

2 

-  20 

CAT. 

2 

10 

3 

-  22 

CAT. 

3 

10 

4 

-  21 

CAT. 

4 

8 

5 

-  25 

CAT. 

5 

4 

6 

-  23 

CAT. 

6 

5 

7 

-  21 

CAT. 

7 

7 

8 

-  22 

CAT. 

8 

6 

9 

-  25 

CAT. 

9 

1 

0 

-  22 

CAT. 

10 

5 

222 

65 

3.  Were  the  verbal  commands  given  during  the  practice  exercises  understand¬ 
able?  (Cassette  recordings) 


Not 

Understandable  Understandable 


Easily 

Understood 


CAT. 

1 

-  2 

CAT. 

1 

-  15 

CAT. 

1  - 

15 

CAT. 

2 

-  1 

CAT. 

2 

-  13 

CAT. 

2  - 

13 

CAT. 

3 

-  1 

CAT. 

3 

-  17 

CAT. 

3  - 

12 

CAT. 

4 

-  4 

CAT. 

4 

-  19 

CAT. 

4  - 

7 

CAT. 

5 

-  1 

CAT. 

5 

-  21 

CAT. 

5  - 

7 

CAT. 

6 

-  3 

CAT. 

6 

-  17 

CAT. 

6  - 

7 

CAT. 

7 

-  4 

CAT. 

7 

-  19 

CAT. 

7  - 

7 

CAT. 

8 

-  5 

CAT. 

8 

-  17 

CAT. 

8  - 

7 

CAT. 

9 

-  1 

CAT. 

9 

-  22 

CAT. 

9  - 

3 

CAT. 

10 

-  2 

24 

CAT. 

10 

-  17 
177 

CAT. 

10  - 

8 

86 

4.  Was  the  timing  of  the  commands  in  the  practice  exercise  spaced 
appropriately? 


Not  Enough 

Enough 

Too 

Much 

Time  Given 

Time  Given 

Time 

Given 

CAT. 

1  -  0 

CAT.  1  -  18 

CAT. 

1  -  11 

CAT. 

2-0 

CAT.  2  -  20 

CAT. 

2-10 

CAT. 

3  -  0 

CAT.  3-17 

CAT. 

3-15 

(Table  12  continued  on  following  page)-- 


Table  12  (continued) — 


CAT. 

4  -  1 

CAT. 

4  - 

17 

CAT. 

4  - 

12 

CAT. 

5  -  0 

CAT. 

5  - 

17 

CAT. 

5  - 

12 

CAT. 

6-0 

CAT. 

6  - 

18 

CAT. 

6  - 

10 

CAT. 

7-0 

CAT. 

7  - 

20 

CAT. 

7  - 

9 

CAT. 

8-0 

CAT. 

8  - 

22 

CAT. 

8  - 

7 

CAT. 

9  -  1 

CAT. 

9  - 

20 

CAT. 

9  - 

6 

CAT. 

10  -  1 

CAT. 

10  - 

18 

CAT. 

10  - 

9 
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5.  Did  you  note  any  possible  errors  in  the  commands  given  during  practice 
exercise?  (Please  list  errors  below.) 

6.  Feel  free  to  make  any  comments  or  suggestions  that  you  think  would 
improve  the  methods  employed  in  this  practice  module. 

COMMENTS  OR  SUGGESTIONS: 


*  ★  * 


Table  13.  Pilot  Study  Evaluation  for  Keyboard 

Circle  the  response  that  best  expresses  your  observations  or  make  comments 
where  appropriate.  The  percentages  for  each  response  are  indicated  in 
parentheses. 

1.  Did  you  find  the  color  scheme  helpful  in  locating  the  keys? 

Not  Helpful  He! pful  Very  Helpful  No  Pi f ference 

0  14  10  4 

(.50)  (.36)  (.14) 

2.  What  is  your  impression  of  how  the  keyboard  is  set  up  in  regard  to  the 
following? 


Not  Helpful 

A.  Flight  ID  Keys  0  (0) 

(orange) 

(Table  13  continued  on  following  page) — 


Hel pful  Very  Helpful 

20  (.71)  8  (.29) 
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B. 

Function  Keys 
(blue) 

1  (.04) 

18  (.64) 

9  (.32) 

C. 

Action  Keys 
(dark  grey) 

3  (.11) 

18  (.64) 

7  (.25) 

D. 

Display  Keys 
(green) 

5  (.18) 

17  (.61) 

6  (.21) 

E. 

Alpha  Keys 
(light  grey) 

6  (.21) 

16  (.57) 

6  (.21) 

F. 

Entry  Keys 
(black)  1  blank 

4  (.14) 

16  (.57) 

8  (.29) 

G. 

Numeric/ Direction  Keys 
(yellow) 

7  (.25) 

X=77T3T 

11  (.39) 

X=T759T 

10  (.36) 

X=T7287 

3.  Do  you  think  that  the  ALPHA  keys  would  be  more  efficient  if  they  were 
arranged  in  the  same  order  as  on  a  typewriter? 

No  Maybe  Yes  No  Difference 

8  (.29)  7(715)  10  (.36)  3  (.10) 

4.  Do  you  think  that  the  NUMERIC/DIRECTION  keys  would  be  more  efficient  if 
they  were  arranged  in  the  same  order  as  on  a  10-key  adding  machine? 

No  Maybe  Yes  No  Di f ference 

9  (.32)  4(7T4)  10  (.36)  5  (.18) 

5.  Which  do  you  feel  would  be  the  best  arrangement  for  the  FLIGHT  ID  keys? 
In  Order  (9)  (.32)  Alternated  (19)  (.68) 


(Table  13  continued  on  following  page) 


Table  13  continued — 


6.  Do  you  think  It  would  be  helpful  if  the  ACTION  keys  were  positioned  on  the 
right  side  and  the  ALPHA  keys  were  positioned  on  the  left  side? 

Not  Helpful  Helpful  Very  Helpful  No  Difference 
12  (.43)  “  3  (.11)  2  (.07)  ll  (.39) 

7.  Do  you  think  the  position  of  the  ENTER  key  is  best  where  it  is? 

Better  Somewhere  Else  (Where?)  No  Difference  Best  There 
10  (.36)  6  (.21)  12  (.43) 


8.  Do  you  think  the  FUNCTION  keys  are  better  at  the  bottom  or  the  top  of 
the  keyboard? 


1  blank  Bottom 

Top 

Better 

Somewhere  Else  (Where?) 

(.04)  2  (.07) 

25  (.89) 

ow 

9.  Was  it  helpful  having  two  SPACE  keys? 

Not  Helpful  Helpful 

Very  Helpful  No 

Difference 

18  (.64)  “  2  (.07) 

0 

8  (.29) 

10.  Did  the  non-functional 

keys : 

No 

No 

Sometimes 

Yes 

Difference 

A.  Cause  you  to  make 

mistakes? 

18  (.64) 

7  (.25) 

0  (0) 

3  (.11) 

B.  Slow  you  down? 

1  blank 

17  (.61) 

7  (.25) 

0  (0) 

4  (.14) 

11.  Do  you  think  that  the  CARRIAGE  RETURN  key  should  be  moved? 

3  blank  No  Yes  No  Difference 

(.11)  4  p4)  2  TT07)  19  (.68) 

12.  Would  it  be  helpful  to  have  the  ACTION  keys  further  apart? 

Not  Helpful  Helpful  Very  Helpful  No  Difference 
10  (.36)  5  (.18)  0  (0)  13  (.46) 

13.  Is  it  confusing  having  the  MIN  ("),  MINI,  and  MAX  ACTION  keys  right  next 
to  each  other? 

No  Sometimes  Yes  No  Difference 

14  (.50)  6  (.21)  5  (.18)"  3  (.11) 

(Table  13  continued  on  following  page)— 


Table  13  conti nued-- 

14.  Were  the  ENTRY  keys  helpful  in  correcting  your  mistakes? 

Not  Helpful  Helpful  Very  Helpful  No  Difference 

6  ( .21 )  10  (.36|  7  (  .25)  5  (.18) 

15.  Which  of  the  following  keys  did  you  use  to  correct  mistakes? 

Never  Rarely  Sometimes  Mostly 

A.  Delete  Character  0  (0)  0  (0)  15  (.58)  11  (.42) 

B.  Delete  Line  0  (0)  1  (.03)  14  (.52)  12  (.45) 

C.  Clear  Display  3  (.12)  10  (.39)  11  (.42)  2  (.07) 

16.  Would  it  be  helpful  to  have  the  DIR  ACTION  key  closer  to  the  NUMERIC/ 
DIRECTION  keys? 

Not  Helpful  Helpful  Very  Helpful  No  Difference 

5  (.18)“'  11  (.40)  6  (.21)”  6  (.21) 


★  *  ★ 


Table  14.  Source  Table  for  Multi  factor  Anova 


Source 

DF 

F 

P 

Main  Effects 

Groups  (G) 

2 

20.724 

.000006 

Notes/No  Notes  (N) 

1 

.002 

.9647 

Short/Long 

Academics  (A) 

1 

.429 

.5187 

2 

.114 

.8927 

2 

.139 

.8709 

1 

.017 

.8973 

5 

.100 

.9912 

Interaction  Effects 


TEMPTS 


MEAN  NUMBER  OF  ERRORS 


'.W 

0.00  1.00  2.00  3.00  4.00  5.00  6.00 

EVALUATION  PROBLEMS 


FIGURE  14.  Errors  on  evaluation  problems. 
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NUMBER  OF  ERRORS 


CATEGORIES  I  THROUGH  10 

FIGURE  15.  Errors  by  category  (evaluation  problem  1). 


FIGURE  16.  Errors  by  category  (evaluation  problem  2). 
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NUMBER  OF  ERRORS  NUMBER  OF  ERRORS  NUMBER  OF  ERRORS 


FIGURE  17.  Errors  by  category  (evaluation  problem  3). 


FIGURE  18.  Errors  by  category  (evaluation  problem  4). 


CATEGORIES  I  THROUGH  10 

FIGURE  19.  Errors  by  category  (evaluation  problem  5). 


3; 


8.00 


7.00 


6.00 


S  5-00 
»- 

Ll 

o 

cr  4.00 

LlI 

GD 

2 

i  3.00 


2.00 


1.00 


0.00 1-1  I - 1 - 1 - 1 - 1 

0.00  1.00  2.00  3.00  4.00  5.00  6.00 

EVALUATION  PROBLEMS 


FIGURE  20.  Number  of  trials  on  evaluation  problems. 


Group  I  - 

Group  n  •  • 

Group  nr  »■  -  # 
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certain  group  and  raw  score  performance.  The  remaining  correlations  of 
attempts,  errors,  and  raw  scores  are  low  and  demonstrate  little  significance. 

B.  Selection  Test  Analysis.  The  model  presented  in  Table  11  demonstrates 
that,  in  combination,  the  Dial  Reading  and  Directional  Headings  tests  are  very 
accurate  in  predicting  performance  on  the  pilot  position  (multiple  R  = 

0.6921).  Consequently,  it  is  suggested  that  these  two  tests  serve  as  a 
battery  for  selecting  outside  hires  to  operate  the  pilot  position.  The  betas 
for  the  two  tests  in  Table  11  are  0.4389  for  the  DRT  and  0.2889  for  the  DHT. 

The  suggested  unit  weights  for  the  two  tests,  based  on  their  betas,  are  5  for 
the  DRT  and  3  for  the  DHT. 

Table  15  demonstrates  the  utility  of  selection  cut  points  of  200  for  the 
community  group  and  150  for  the  handicapped  group  on  the  Dial  Reading  and 
Directional  Headings  composite  score.  Only  four  subjects  from  the  community 
group  whose  raw  score  was  at  or  below  40,  scored  above  the  cutoff  on  the 
composite.  A  lower  cut  point  was  assigned  to  the  handicapped  persons  since  a 
cut  point  of  200  eliminates  five  more  of  the  handicapped  than  does  the  150  cut 
point.  Further,  it  can  be  noted  that  the  mean  performance  of  the  subjects 
above  the  cut  points  is  equivalent  to  the  mean  performance  of  the  most 
proficient  group.  Group  I.  (See  Table  15  and  Figure  13.) 

C.  Practice  Problems  Evaluating  Form.  Item  1  in  Table  12  shows  that  63 
percent  of  the  subjects  stated  that  the  commands  were  taught  very  clearly,  36 
percent  stated  that  the  commands  were  taught  with  average  clarity,  while  only 

1  percent  stated  that  the  commands  (category  9)  were  taught  unclearly.  Item  2 
of  that  same  questionnaire  states  that  77  percent  of  the  subjects  felt  that 
enough  time  was  allowed  for  the  dry  runs.  The  responses  in  item  3  indicate 
that  the  verbal  commands  given  during  practice  exercises  could  be  improved  but 
basically  were  understandable  (62  percent),  30  percent  said  the  commands  were 
not  easily  understood,  and  8  percent  said  the  commands  could  not  be  understood. 
In  it  cm  4  the  responses  indicate  that  64  percent  felt  that  enough  time  was 
given  in  the  practice  exercises  for  entering  messages  while  35  percent  felt 
that  too  much  time  was  given. 

D.  Keyboard  Evaluation  Form.  Table  13  indicates  that  the  subjects  felt 
the  color  scheme  and  the  genera)  keyboard  setup  were  "helpful"  to  "very 
helpful."  Fifty  percent  viewed  the  color  scheme  helpful,  and  36  percent 
viewed  the  color  scheme  as  very  helpful,  while  59  percent  viewed  the  keyboard 
setup  as  helpful  and  28  percent  viewed  the  keyboard  setup  as  very  helpful. 
Suggested  changes  to  the  keyboard  listed  in  items  18  and  19  (see  Appendix  4) 
of  the  keyboard  evaluation  form  were:  (i)  the  DIR  key  should  be  moved  nearer 
to  the  direction/numeric  key  pad  since  it  is  only  used  in  conjunction  with  this 
key  pad,  (1i)  the  extra  space  key  was  not  helpful  and  should  be  put  to  better 
use,  and  (iii)  a  DELETE  key  to  correct  inputs  made  on  the  function  keys  should 
be  provided.  At  present  it  Is  impossible  to  delete  an  entire  function  since 
the  DELETE  key  operates  on  one  character  at  a  time. 
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Table  15.  Graphic  Representation  of  Selection  Cut  Points 


Raw  Scores 

Selection  Composite 

Group 

39 

56 

3 

39 

105 

3 

40 

115 

3 

44 

121 

2 

Cut  Point  for 

40 

122 

3 

Handicapped^  Group_ 

30 

140 

3  150  Cut  Point 

Raw  Score  X=44.42 

40 

169 

3 

Cut  Point  for 

48 

175 

2  200  Cut  Point 

Community  jlroup 

45 

205 

2 

Raw  Score  X=45 .12 

40 

209 

3 

40 

210 

3 

42 

214 

2 

44 

227 

2 

43 

247 

2 

44 

260 

3 

48 

268 

2 

43 

274 

3 

42 

279 

1 

47 

289 

2 

43 

291 

3 

47 

294 

1 

47 

310 

2 

43 

327 

1 

40 

347 

1 

48 

360 

1 

41 

369 

1 

48 

372 

2 

43 

400 

2 

48 

406 

1 

46 

411 

2 

49 

451 

1 

49 

453 

1 

48 

476 

1 

47 

499 

1 

E.  Multi  factor  Analyses  of  Variance.  The  ANOVA  source  table  (Table  14) 
demonstrates  only  one  significant  independent  effect,  vi z  group  membership. 
The  Academy  graduate  group  and  the  community  group  performed  essentially  the 
same;  however,  the  latter  had  72  hours  of  training  as  opposed  to  40  hours  of 
training  for  the  Academy  graduate  group.  Post  hoc  analyses  showed  that  the 
handicapped  group  achieved  proficiency  with  80  hours  of  training  but  was 
significantly  lower  in  performance  than  both  Groups  I  and  II.  (See  means  for 
I,  II,  and  III  in  Figures  12,  13,  and  14.)  The  ANOVA  shows  that  it  made  no 
difference  in  performance  whether  subjects  were  allowed  to  take  notes  or  not. 
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or  whether  they  were  given  "long"  or  "short"  academic  preparation.  There  were 
no  significant  interaction  effects  between  any  of  the  three  independent 
variables  taken  two  at  a  time  or  three  at  a  time. 

F.  Attempts,  Errors,  Raw  Scores,  and  Number  of  Trials  Across  Evaluation 
Exercises  and  Command  Categories.  In  Figure  12  the  number  of  attempts 
steadily  decremented  from  evaluation  exercise  1  to  5.  The  largest  number  of 
attempts  was  made  by  Group  II  on  evaluation  exercise  1.  The  lowest  number  of 
attempts  was  made  by  Group  III  on  evaluation  exercise  5.  These  results  could 
be  interpreted  in  one  of  several  ways.  First,  it  could  mean  that  since  less 
time  is  allowed  from  evaluation  exercise  1  to  5,  there  was  less  opportunity  to 
make  attempts.  Second,  it  could  mean  that  the  inputs  became  more  accurate. 
More  than  likely,  the  results  reflect  a  combination  of  both  opportunity  and 
accuracy. 

Errors  on  evaluation  exercises  also  show  a  steady  decrease  from  evaluation 
exercise  1  through  5.  The  largest  number  of  errors  was  made  by  Group  II  on 
evaluation  exercise  1.  The  smallest  number  of  errors  was  made  by  Group  I  on 
evaluation  exercise  5.  Overall,  Group  I  had  the  least  errors  across  the 
evaluation  exercises,  Group  II  was  second,  and  Group  III  was  third  in 
accuracy.  (See  Figure  13.) 

Raw  scores  follow  the  same  pattern  as  do  errors,  with  Group  I  having  the 
highest  scores.  Group  II  second  highest,  and  Group  III  the  lowest.  (See 
Figure  13.) 

The  number  of  trials  on  evaluation  exercises  was  largest  for  Group  III, 
next  largest  for  Group  II,  and  smallest  for  Group  I.  It  required  about  40 
hours  of  "hands  on"  training  time  for  Group  I,  72  hours  for  Group  II,  and 
about  88  hours  for  Group  III.  (See  Table  16.) 

Errors  across  all  categories  decreased  from  evaluation  exercise  1  to  5. 

The  largest  number  of  errors  was  made  on  categories  2,  3,  6,  and  9  by  all 
three  groups  across  all  five  evaluation  exercises.  It  appears  from  viewing 
the  evaluation  exercise  that  categories  2  and  3  contain  too  many  commands  to 
be  taught  compared  to  the  other  command  categories.  There  were  16  commands  in 
category  2  and  10  in  category  3.  Items  in  categories  6  and  9  offer  no  clue  as 
to  why  the  commands  were  difficult.  (See  Figures  15,  16,  17,  18,  and  19.) 

It  is  concluded  from  the  data  on  attempts,  errors,  raw  scores,  and  number 
of  trials  across  evaluation  problems  that  Groups  I,  II,  and  III  can  all  be 
trained  to  the  proficiency  level  required  to  operate  the  RTF  problems.  The 
ratio  of  training  time  required  in  this  study  was  about  2:1;  i.e.,  it 
required  approximately  twice  as  long  to  train  community  and  handicapped 
subjects  to  the  required  proficiency  level.  It  seems  reasonable  to  assume 
that  more  extensive  training  for  the  handicapped  and  community  groups  would 
result  in  a  higher  proficiency  level  than  that  achieved  in  this  study.  If 
Academy  students  are  employed  to  operate  the  pilot  position,  each  input  would 
be  trained  only  to  a  minimum  level  of  proficiency  because  a  new  group  of 
trainees  would  be  trained  every  input. 
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Table  16.  Training  Time  Required  by  Each  Group 


Trials 


Groups 

Hours  in  Training 

Practice  Problems 

Evaluation  Problems 

I 

40 

16 

8 

II 

72 

24 

11 

III 

88 

17 

14 

G.  Evaluation  of  Training  System.  All  three  groups  met  the  training 
proficiency  requirements  for  operating  the  pilot  position  within  a  reasonable 
length  of  time.  Verbal  statements  from  the  FAA's  National  Aviation  Facilities 
Experimental  Center,  Chicago-O'Hare  Tower,  and  the  Hull  Center  in  Ottawa, 
Canada,  indicate  8  to  12  weeks  on  the  average  for  training  time.  The  data  from 
the  present  study  support  the  feasibility  of  using  computer-assisted  instruc¬ 
tion  to  teach  the  pilot  position  in  much  less  time  (see  Table  16)  than  the 
reported  training  times  where  no  programed  instruction  is  employed.  Therefore, 
we  strongly  recommend  the  use  of  computer-assisted  training  and  evaluation  for 
the  pilot  position.  Using  the  computer  to  teach  this  position  not  only 
reduces  training  time,  it  also  helps  make  maximum  use  of  the  computer  system. 
Since  the  tasks  involved  in  operating  the  ghost  position,  the  Plan  View 
Display  (PVD),  and  the  Data  Entry  and  Display  (f  )  are  so  similar  to  the 
operation  of  the  pilot  position,  it  seems  sen$*b'e  to  generalize  the  results  of 
this  study  to  the  methods  of  teaching  these  other  position  keyboards  also. 
Consequently,  we  further  suggest  that  computer-assisted  instruction  be 
employed  to  teach  these  positions  also.  A  functional  description  of  a 
computer-assisted  teaching  method  for  keyboards  is  in  Appendix  5. 

Several  problems  were  identified  in  the  prototype  pilot  training  system 
used  in  this  study  that  need  to  be  corrected  before  the  system  can  be  employed 
in  computer-assisted  training.  The  major  problem  was  with  the  audio  system. 

It  was  difficult  to  synchronize  the  computer  timing  mechanism  with  the  audio 
output.  The  training  system  program  provided  a  bell  tone  when  the  audio 
inputs  were  to  be  recorded;  however,  in  the  run  mode  the  timing  allowed  by  the 
computer  for  inputting  was  not  consistent  with  the  timing  and  the  bell. 

Further,  even  if  the  bell  had  worked  properly,  it  still  would  have  been 
difficult  to  synchronize  the  audio  with  the  computer  timing  mechanism. 

Another  problem  in  the  auuio  system  was  tone  quality.  The  audio  was  not  clear 
and  the  messages  were  sometimes  difficult  to  understand. 

A  major  software  problem  with  the  present  training  system  was  the  setup 
for  clearing  the  terminals.  The  sequential  CRT  clearing  process  required 
about  5  seconds.  The  person  on  the  first  terminal  had  5  seconds  longer  to 
input  the  command  message  than  did  the  person  in  the  last  position.  A  method 
for  simultaneously  clearing  the  terminals  needs  to  be  developed. 
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Computer-assisted  instruction  and  evaluation  have  been  successfully 
employed  in  many  occupational  fields.  Routine  tasks  requiring  the  operation 
of  machinery  are  especially  adaptable  to  computer-assisted  instruction.  All 
three  groups  employed  in  this  study  performed  well  using  this  learning 
method.  It  would  appear  to  be  a  valuable  asset  to  the  FAA  Academy  in  terms 
of  personnel  utilization  and  maximum  use  of  the  computer  to  employ 
computer-assisted  instruction  in  teaching  the  keyboard  operations.  This  would 
be  true  without  regard  to  whether  handicapped  persons,  community  persons,  or 
Academy  students  were  the  trainees. 

V.  Summary  of  Recommendations. 

A.  Use  of  Academy  Students  Versus  Use  of  Handicapped  and  Community 
Persons  to  Operate  the  Pilot  Positions. 

1.  In  the  absence  of  any  compelling  financial  or  personnel  reasons, 
it  is  suggested  that  community  persons  and  handicapped  persons  be  employed 
for  the  pilot  positions. 

2.  If  handicapped  and  community  persons  are  hired,  they  should  be 
qualified  for  selection  on  the  basis  of  scores  on  the  Dial  Reading  Test  and 
Directional  Headings  Test.  The  composite  of  these  two  tests  should  be  formed 
by  weighting  the  Dial  Reading  Test  5  and  the  Directional  Headings  Test  3.  Two 
different  cut  points  should  be  used  for  community  persons  and  handicapped.  The 
recommended  cut  point  for  community  people  is  200.  The  recommended  cut  point 
for  handicapped  applicants  is  150. 

B.  Use  of  a  Computer-Assisted  Training  System. 

1.  A  computer-assisted  training  system  with  programed  instruction  is 
strongly  recommended  for  use  in  training  persons  to  operate  the  pilot  position, 
ghost  position,  and  PVD  and  DEDS  positions. 

2.  The  prototype  system  used  in  this  study  should  be  implemented  (see 
Appendix  5)  for  keyboard  training  within  the  structure  of  the  present  RTF 
system  configuration. 

3.  An  improved  audio  system  should  be  developed  that  has  better  tone 
quality  and  a  better  means  of  synchronizing  the  computer  operations  with  the 
audio  outputs. 

C.  Changes  to  the  Pilot  Keyboard. 

1.  The  DIR  key  should  be  moved  nearer  to  the  NUMERIC/ DIRECTION 

keypad. 

2.  A  DELETE  key  should  be  provided  to  correct  erroneous  function  key 

entries. 
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3.  A  more  profitable  use  should  be  made  of  the  extra  (green)  space 


key. 


D.  Suggested  Curricula  Development. 

1.  Since  the  data  demonstrated  that  there  was  no  difference  in  perform¬ 
ance  between  those  who  received  "long"  and  those  who  received  "short"  academic 
training,  it  is  recommended  that  only  the  minimal  amount  of  academic  prepara¬ 
tion  required  to  operate  the  position  be  provided,  allowing  more  hands-on 
training  time. 

2.  Since  there  was  no  difference  in  performance  between  those  who  took 
notes  and  those  who  did  not,  note  taking  should  be  optional. 

3.  Command  categories  number  2  and  3  should  be  broken  up  into  smaller 
categories  for  instructional  units. 
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Appendix  1 


CATEGORY  #1 
Command 
Command 
Command 


Command 

Command 

CATEGORY  #2 
Command 

Command 

Command 

Command 

Command 

Command 

Command 

Command 

Command 

Command 

Command 


Command 

Command 


Command 


Command 

Command 

CATEGORY  #3 
Command 


(Appendix  1 


1.  (Flight  ID)  STOP  TURN  (Heading  FK  Enter) 

2.  (Flight  ID)  DISPLAY  ALL  FLIGHT  PLAN  DATA  (Display  FK  Enter) 

3.  (Flight  ID)  CONTACT  Memphis  CENTER  ONE  TWO  NINER  POINT  ZERO 

Kansas  City  CENTER  ONE  TWO  SEVEN  POINT  ZERO 
Albuquerque  CENTER  ONE  TWO  SIX  POINT  ZERO 
(Frequency  FK  Enter) 

4.  (Flight  ID)  CLEARED  AS  FILED  (Depart  FK  Enter) 

5.  (Flight  ID)  MAINTAIN  PRESENT  SPEED  (Speed  FK  Enter) 


1.  (Flight  ID)  FLY  HEADING  TWO  FIVE  ZERO  (Heading  FK  250  Enter) 

(Headings  are  changed  on  dry  runs) 

2.  (Flight  ID)  MAINTAIN  ONE  FOUR  THOUSAND  (Altitude  FK  140  Enter) 

(Altitudes  are  changed  on  dry  runs) 

3.  (Flight  ID)  REDUCE  SPEED  TO  TWO  FOUR  ZERO  KNOTS  (Speed  FK  240 

Enter)  (Speeds  are  changed  on  dry  runs) 

4.  (Flight  ID)  REDUCE  TO  HOLDING  SPEED  (Speed  FK  H  Enter) 

5.  (Right  ID)  RESUME  NORMAL  SPEED  (Speed  FK  RZM  Enter) 

6.  (Right  ID)  FLY  RUNWAY  HEADING  (Heading  FK  R  Enter) 

7.  (Right  ID)  IDENT  (Beacon  FK  I  Enter) 

8.  (Flight  ID)  TURN  RIGHT  (Heading  FK  -  Enter) 

(Right  has  been  interchanged  with  left  on  dry  runs) 

9.  (Flight  ID)  REDUCE  TO  APPROACH  SPEED  (Speed  FK  A  Enter) 

10.  (Flight  ID)  CLEARED  VIA  LAST  ROUTING  CLEARED  (Route  FK  RZM 

Enter) 

11.  (Flight  ID)  SQUAWK  ALTITUDE  (Beacon  FK  A  Enter) 

STANDBY  (Beacon  FK  S  Enter)  (Altitude  has  been  inter¬ 
changed  with  standby  on  dry  runs) 

12.  (Flight  ID)  INTERCEPT  FLIGHT  PLAN  ROUTE  (Route  FK  =,  Enter) 

13.  (Flight  ID)  CLEARED  FOR  NDB  APPROACH  (Approach  FK  N  Enter) 

VOR  APPROACH  (Approach  FK  V  Enter) 

ILS  APPROACH  (Approach  FK  I  Enter) 

14.  (Flight  ID)  CLEARED  TO  TULSA  (Clearance  Limit  FK  TUL  Enter) 

McALESTER  (Clearance  Limit  FK  MLC  Enter)  (TUL  and  MLC  are 

interchanged  on  dry  runs) 

15.  (Right  ID)  EXPEDITE  TURN  (Heading  FK  Max  Key  Enter) 

16.  (Flight  ID)  SQUAWK  TWO  FIVE  ZERO  TWO  (Beacon  FK  2502  Enter) 


1.  (Flight  ID)  TURN  RIGHT  HEADING  THREE  SIX  ZERO  (Heading  FK  -*■ 
360  Enter)  (Right  and  left  are  interchanged  and  headings 
are  changed  on  the  dry  runs) 

continued  on  following  page)-- 
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Command  2.  (Flight  ID)  CLIMB  AT  PILOT'S  DISCRETION  MAINTAIN  FLIGHT  LEVEL 
TWO  TWO  ZERO  (Altitude  FK  MINI  220  Enter)  (Climb  and 
descend  are  interchanged  and  flight  levels  are  changed  on 
the  dry  runs) 

Command  3.  (Flight  ID)  CRUISE  ONE  ONE  THOUSAND  (Altitude  FK  110  CRZ 
Enter)  (Altitudes  are  changed  on  the  dry  runs) 

Command  4.  (Flight  ID)  DESCEND  IMMEDIATELY  MAINTAIN  SIX  THOUSAND 

(Altitude  FK  MAX  60  Enter)  (Altitudes  are  changed  on  the 
dry  runs) 

Command  5.  (Flight  ID)  CLEARED  FOR  ILS  STRAIGHT  IN  APPROACH  (Approach  FK 
I  (or  V)  S  Enter)  (ILS  and  VOR  are  interchanged  on  the 
dry  runs) 

Command  6.  (Flight  ID)  SQUAWK  TWO  TWO  ZERO  ZERO  AND  I  DENT  (Beacon  FK 
2200  I  Enter)  (Beacon  codes  are  changed  on  the  dry  runs) 

Command  7.  (Flight  ID)  REPORT  REACHING  ONE  SEVEN  THOUSAND  (Report  FK  R 
(or  L)  170  Enter)  (Reaching  and  leaving  are  interchanged 
and  altitudes  are  changed  on  the  dry  runs) 

Command  8.  (Flight  ID)  CLEARANCE  VOID  IF  NOT  OFF  GROUND  BY  ONE  FOUR 

THREE  ZERO  (Depart  FK  >  1430  Enter)  (Times  are  changed  on 
dry  runs) 

Command  9.  (Flight  ID)  TURN  FORTY  DEGREES  RIGHT  (Heading  FK  40  -  Enter) 
(Right  and  left  are  interchanged  and  headings  are  changed 
on  the  dry  runs) 

Command  10.  (Flight  ID)  RADAR  CONTACT  (LOST)  (Report  FK  R  C  (or  L) 

Enter)  (Radar  contact  and  radar  contact  lost  are  inter¬ 
changed  on  the  dry  runs) 

CATEGORY  #4 

Command  1.  (Flight  ID)  MAINTAIN  PRESENT  HEADING  UNTIL  REACHING  ONE  ONE 
THOUSAND  (Heading  FK  /  Altitude  FK  110  Enter)  (Altitudes 
are  changed  on  the  dry  runs) 

Command  2.  (Flight  ID)  CONTACT  Tulsa  approach  ONE  ONE  NINER  POINT  TWO  AT 
Mayes  (Freq  FK  @  FRD  FK  Mayes  Enter)  (Tulsa  approach  and 
Mayes  are  interchanged  with  these:  Memphis  Center,  Miami; 
and  Fort  Worth  Center,  McAl ester  on  the  dry  runs) 

Command  3.  (Flight  ID)  CLEARED  FOR  BOLDE  ONE  DEPARTURE  (Depart  FK  (SID) 
Boldel  Enter) 

Command  4.  (Flight  ID)  MAINTAIN  HEADING  ONE  FIVE  ZERO  UNTIL  TWO  TWO  ONE 
TWO  (Heading  FK  1 50  /  2212  Enter)  (Headings  and  times  are 
changed  in  the  dry  runs) 

Command  5.  (Flight  ID)  DESCEND  AND  MAINTAIN  FLIGHT  LEVEL  TWO  EIGHT  ZERO 
AT  TWO  ONE  THREE  ZERO  (Altitude  FK  280  9  2130  Enter) 

(Climb  and  descend  are  interchanged  and  the  times  and 
altitudes  are  changed  in  the  dry  runs) 

(Appendix  1  continued  on  following  page)— 
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CATEGORY  #5 

Command  1.  (Flight  ID)  MAINTAIN  HEADING  ZERO  NINER  ZERO  UNTIL  REACHING 
ONE  TWO  THOUSAND  (Heading  FK  090  /  Altitude  FK  120  Enter) 
(Altitudes  and  headings  are  changed  on  the  dry  runs) 

Command  2.  (Flight  ID)  MAINTAIN  TWO  TWO  ZERO  KNOTS  UNTIL  TULSA  (Speed 
FK  220  /  FRO  FK  Tul  Enter)  (Speed  has  changed  and  Tulsa 
has  been  interchanged  with  Miami  in  the  dry  runs) 

Command  3.  (Flight  ID)  DEPART  MIAMI  HEADING  THREE  SIX  ZERO  (Depart  FK 
FRD  FK  MIO  Heading  FK  360  Enter)  (Miami  has  been  inter¬ 
changed  with  Tulsa  and  the  headings  have  been  changed  on 
the  dry  runs) 

CATEGORY  #6 

Command  1.  (Flight  ID)  CLEARED  TO  McALESTER  HOLD  NORTH  LEFT  TURNS 
(Clearance  Limit  FK  MLC  Hold  FK  Dir  Key  N  «-  Enter) 

(McAl ester  has  been  interchanged  with  Tulsa  and  Miami  and 
the  directions  and  left  and  right  have  been  changed  on  the 
dry  runs) 

Command  2.  (Flight  ID)  FLY  HEADING  ONE  EIGHT  ZERO  UNTIL  REACHING  ONE 
TWO  THOUSAND  BEFORE  PROCEEDING  ON  COURSE  (Heading  180  / 
Altitude  FK  120  RZM  Enter)  (Headings  and  altitudes  have 
been  changed  on  dry  runs) 

CATEGORY  #7 

Command  1.  (Flight  ID)  CLEARED  VIA  BOLDE  ONE  DEPARTURE  SPRINGFIELD 

TRANSITION  (Depart  FK  (SID)  Bolde  1  T  SGFEnter)  (Spring- 
field  is  interchanged  with  Fort  Smith  on  dry  runs) 

Command  2.  (Flight  ID)  HOLD  SOUTH  OF  MIAMI  ON  THE  ONE  EIGHT  ZERO  RADIAL 
TWO  MINUTE  LEGS  (Hold  FK  Dir  Key  S  FRD  FK  MIO  2  Min  Enter) 
(Direction  and  minutes  have  been  changed  and  Miami  has 
been  interchanged  with  McAl ester  and  Tulsa  on  the  dry  runs) 

CATEGORY  #8 

Command  1.  (Flight  ID)  CROSS  TULSA  AT  ONE  ONE  THOUSAND  CLEARED  FOR  VOR 
APPROACH  (Altitude  FK  X  FRD  TUL  @  110  Approach  FK  V  Enter) 
(Altitudes  are  changed  and  TUL  in  interchanged  with  MLC  and 
MIO  on  the  dry  runs. ) 

CATEGORY  #9 

Command  1.  (Flight  ID)  CLEARED  FOR  TULSA  ONE  ARRIVAL  CROSS  TULSA  AT  OR 
ABOVE  ONE  THREE  THOUSAND  (Route  FK  (STAR)  TULSA1  X  FRD  RK 
TUL  t  Altitude  FK  130  Enter)  (Altitudes  are  changed  on  the 
dry  runs) 

Command  2.  (Flight  ID)  CLEARED  VIA  BOLDE  ONE  DEPARTURE  CROSS  INOLA  AT 
OR  BELOW  SEVEN  THOUSAND  (Depart  FK  (SID)  Bolde  1  X  FRD  FK 
INOLA  f  Altitude  FK  70  Enter)  (Inola  has  been  inter¬ 
changed  with  Flint  and  Bolde  and  altitudes  have  been 
changed  on  dry  runs) 

(Appendix  1  continued  on  following  page)-- 
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CATEGORY  #10 

Command  1.  (Flight  ID)  HOLD  EAST  OF  MIAMI  ON  VICTOR  TWO  THREE  MINUTE 

LEGS  RIGHT  TURNS  (Hold  FK  Dir  Key  E  FRD  FK  MIO  V2  3  Min  -*• 
Enter)  (Victor  #s,  times,  directions,  places,  and  right  or 
left  have  been  changed  on  the  dry  runs) 

Command  2.  (Flight  ID)  CROSS  VICTOR  THREE  TWO  ZERO  MILES  EAST  OF  TULSA 

(Route  FK  X  V3  20  miles  Dir  Key  E  FRD  FK  TUL  Enter)  (Victor 
#s,  directions,  and  places  have  been  changed  on  dry  runs) 


I 
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Appendix  2 

BIOGRAPHICAL  QUESTIONNAIRE 

All  the  items  which  follow  are  in  the  familiar  multiple  choice  format. 

Answer  each  one  by  blackening  the  circle  in  the  appropriate  column  (A,  B,  C, 
D,  or  E)  on  your  answer  sheet.  Choose  the  response  that  best  fits  you  and 
only  make  one  response  per  question. 

HIGH  SCHOOL  EDUCATION 


1.  Which  of  the  following  best  describes  your  high  school  career? 


A.  Did  not  attend  high  school 

B.  Did  not  complete  high  school 

C.  High  school  diploma  granted  by  school 

D.  High  school  diploma  granted  by  G.E.D. 

2.  How  old  were  you  when  you  left  high  school? 


A.  15  or  younger 

B.  16 

C.  17 

D.  18 

E.  19  or  older 


What  grades,  on  the  average,  did  you  get  in  the  following  high  school 
courses?  Fill  in  the  letter  corresponding  to  the  grade  for  each  subject. 

A.  About  "A-"  to  "A+" 

B.  About  "B-"  to  "B+" 

C.  About  "C-"  to  "C+" 

D.  Lower  than  "C-" 

E.  Did  not  have  course 


3.  Arithmetic,  Math 

4.  Physical  Science 

5.  Biological  Science 

6.  English 

7.  Social  Studies 


(Appendix  2  continued  on  following  page) 
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8.  Business  or  Commercial 

9.  Physical  Education 

10.  Vocational  or  Agricultural 

11.  What  was  your  overall  high  school  average? 

12.  When  you  were  growing  up,  about  how  many  books  were  around  the  house? 

A.  A  large  library 

B.  Several  bookcases  full 

C.  One  bookcase  full 

D.  A  shelf  full 

E.  Very  few  or  none 

13.  While  in  high  school,  how  many  of  the  following  positions  did  you  hold? 

Chairperson  of  an  important  student  committee 

Cheerleader 

Class  officer 

Editor  of  a  publication 

Leading  actor  in  a  play 

Member  of  the  student  council 

Member  of  the  debating  team 

President  of  an  honorary  scholastic  organization 
Speaker  at  the  class  commencement 
Captain  of  an  athletic  team 
President  of  a  student  club 

A.  0  to  2 

B.  3  or  4 

C.  5  or  6 

D.  7  or  8 

E.  9  to  11 

14.  Relative  to  your  close  friends,  how  well  did  you  do  in  physical  or 
athletic  activities  in  high  school? 

A.  Much  better  than  they  did 

B.  Somewhat  better  than  they  did 

C.  About  as  well  as  they  did 

D.  Not  quite  as  well  as  they  did 

E.  Not  nearly  as  well  as  they  did 

(Appendix  2  continued  on  following  page)— 
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15.  In  high  school,  when  friends  came  to  you  with  their  personal  problems, 
how  likely  were  you  to  go  out  of  your  way  to  give  them  help  or  advice? 


A.  Much  more  likely  than  most  people 

B.  Somewhat  more  likely  than  most  people 

C.  About  as  likely  as  others 

D.  Somewhat  less  likely  than  most  people 

E.  Much  less  likely  than  most  people 

16.  Before  or  during  high  school,  did  you  ever  conduct  a  scientific  experi¬ 
ment  on  your  own  initiative  (not  as  part  of  any  required  school 
assignment)? 

A.  No 

B.  Yes-both  before  and  during  high  school 

C.  Yes-before  high  school 

D.  Yes-during  high  school 

EDUCATION  BEYOND  HIGH  SCHOOL 

17.  Which  of  the  following  best  describes  your  jcational  background? 

A.  No  formal  education  beyond  high  school 

B.  Attended  college  but  did  not  receive  a  degree 

C.  Attended  college  and  received  a  degree 

D.  Have  done  college  graduate  work,  but  did  not  receive  a  degree 
beyond  a  bachelor 

E.  Have  an  advanced  degree  (M.S.,  M.A.,  Ph.D. ,  etc.) 

18.  How  long  has  it  been  since  you  last  attended  school  as  a  full  time 
student? 

A.  Less  than  a  year 

B.  1-2  years 

C.  2-3  years 

D.  3-4  years 

E.  Over  4  years 

From  the  following  three  questions,  answer  the  one  that  is  appropriate, 
either  item  19,  20,  or  21. 

What  was  your  major  area  of  study  in  college? 
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19.  A.  Did  not  attend  college 

B.  Dramatic  Arts 

C.  Languages 

D.  Humanities 

E.  Biological  Science 

20.  A.  Physical  Science 

B.  Social  Science 

C.  Applied  Studies  (Agriculture) 

D.  Applied  Studies  (Business) 

E.  Applied  Studies  (Education) 

21.  A.  Applied  Studies  (Home  Economics) 

B.  Applied  Studies  (Speech,  Journalism) 

C.  Applied  Studies  (Professional) 

D.  Doubl e  Major 

E.  Other 

22.  Before  appointment  as  an  ATC  specialist  how  often  had  you  flown  in  an 
ai rplane? 

A.  Very  often 

B.  Often 

C.  Sometimes 

D.  Seldom 

E.  Never 

23.  Before  appointment  as  an  ATC  specialist  how  much  time  had  you  spent 
around  airports? 

A.  Very  much 

B.  Much 

C.  Some 

D.  Little 

E.  Very  little 

MILITARY  EXPERIENCE 

24.  Do  you  have  prior  military  experience? 


A. 

No  (if  no,  skip  #25) 

Which 

branch  of  the  service? 

A. 

USAF 

B. 

USN 

C. 

USMC 

D. 

USA 

E. 

USCG 

(Appendix  2  continued  on  following  page)— 


26.  Do  you  have  a  prior  control  tower  operator  (CTO)  rating? 


A.  No 

B.  Yes-IFR 

C.  Yes-VFR 

D.  Yes-both  IFR  and  VFR 

27.  Do  you  have  a  prior  air  traffic  control  specialist  rating? 

A.  No 

B.  Yes-Center 

C.  Yes-Flight  Service  Station  (FSS) 

D.  Yes-both  Center  and  FSS 

28.  Do  you  have  prior  IFR  operation  experience? 

A.  No 

B.  Yes-military 

C.  Yes-civilian 

D.  Yes-military  and  civilian 

For  the  following  types  of  IFR,  approximate  the  amount  of  experience  you 
had  using  the  following  scale: 

A  =  None 

B  =  Under  6  months 
C  =  6  to  12  months 
D  =  12  to  18  months 
E  =  Over  18  months 

29.  ARTCC 

30.  RATCC  or  CATCC 

31 .  ARAC 

32.  RAP CON 

33.  TOWER 

34.  GCA  (RADAR) 

35.  GCI  (RADAR) 

36.  Have  you  had  prior  VFR  operations  experience? 
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A.  No 

B.  Yes-military 

C.  Yes-civilian 

D.  Yes-military  and  civilian 

Use  the  following  scale  to  describe  the  amount  of  VFR  operations  experience 
you  have  had  for  the  types  listed  below. 

A  =  None 

B  =  Under  6  months 
C  =  6  to  12  months 
D  =  12  to  18  months 
E  =  Over  1 3  months 

37.  Tower 

38.  FSS  or  IFSS 

39.  GCI  (nonradar) 

Use  the  scale  below  to  list  tjj$  i>rior  communications  operations  experience  you 
have  had  for  the  items  beli?w. 

A  =  None 

B  =  Restricted  rating 
C  =  3rd  class  rating 
D  =  2nd  class  rating 
E  =  1st  class  rating 

40.  Station 

41  .  Ground  to  air 

43.  Point  to  point 
PRIOR  AIRMAN  CERTIFICATE 

For  the  following  types  of  certificates  or  ratings,  use  the  scales  below  to 
indicate  whether  or  not  you  have  the  certificate  and  if  it  is  based  on  military 
and/or  civilian  experience. 

A  =  No  certificate  or  rating 
B  =  Military  experience 
C  =  Civilian  experience 
D  =  Both  military  and  civilian 

43.  Airl ine  transport 

44.  Commercial 

45.  Private 

46.  Flight  Instructor 

47.  Ground  Instructor 

(Appendix  2  continued  on  following  page)-- 
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48.  Dispatch-air  carrier 

49.  Navigator/bombardier 

50.  Single-engine  jet 

51.  Single-engine  turbo  prop 

52.  Single-engine  reciprocating 

53.  Multi-engine  jet 

54.  Multi-engine  turbo  prop 

55.  Multi-engine  reciprocating 

56.  Instrument 

57.  Rotorcraft 

58.  Glider 

59.  Prior  to  Phase  I  indoctrination,  what  type  of  air  traffic  control 
experience/training  have  you  had  with  FAA? 

A.  FSS 

B.  Terminal 

C.  En  Route 

D.  Predevelopmental  or  other  FAA  preparatory  training 

E.  None 

60.  Prior  to  Phase  I  indoctrination,  how  long  have  you  been  in  the  2152 
option? 

A.  Was  not  in  2152  option  prior  to  Phase  I  indoctrination 

B.  Less  than  1  year 

C.  1-2  years 

D.  3-4  years 

E.  5  years  or  more 
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Appendix  3 
PILOT  STUDY 

Evaluation  Form  for  Practice  Modules. 

Directions:  Circle  the  response  that  best  expresses  your  observation  or  make 
comments,  where  appropriate. 

1.  Were  the  commands  taught  clearly? 

1  3  5 

unclear  average  clear  very  clear 

2.  Was  enough  practice  time  given?  (Dry  runs) 

1  3  5 

not  too 

enough  enough  much 

3.  Were  the  verbal  commands  given  during  the  practice  exercise  understand¬ 
able?  (the  cassette  recordings) 

1  3  5 

not  easily 

understandable  understandable  understood 

4.  Was  the  timing  of  the  commands  in  the  practice  exercise  spaced  appro¬ 
priately? 

1  3  5 

not  enough  too 

enough  time  much 

time  given  given  time  given 

5.  Did  you  note  any  possible  errors  in  the  commands  given  during  the  practice 
exercise?  (Please  list  errors  below.) 


6.  Feel  free  to  make  any  comments  or  suggestions  that  you  think  would  improve 
the  methods  employed  in  this  practice  module. 

Comments  or  Suggestions: 
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Appendix  4 

PILOT  STUDY  EVALUATION  FOR  KEYBOARD 


Circle  the  response  that  best  expresses  your  observations  or  make  comments 

where  appropriate. 

1.  Did  you  find  the  color  scheme  helpful  in  locating  the  keys? 

12  3  4 

Not  Helpful  Helpful  Very  Helpful  No  Difference 

2.  What  is  your  impression  of  how  the  keyboard  is  set  up  in  regard  to: 

Not  Helpful  Helpful  Very  Helpful 

Flight  ID  Keys  1  2  3 

(orange) 

Function  Keys  1  2  3 

(bl  ue) 

Action  Keys  1  2  3 

(dark  grey) 

Display  Keys  1  2  3 

(green) 

Alpha  Keys  1  2  3 

(light  grey) 

Entry  Keys  1  2  3 

(bl ack) 

Numeric/ Di recti  on  Keys  1  2  3 

(yel low) 

3.  Do  you  think  that  the  ALPHA  keys  would  be  more  efficient  if  they  were 

arranged  in  the  same  order  as  on  a  typewriter? 

12  3  4 

No  Maybe  Yes  No  Difference 

4.  Do  you  think  that  the  NUMERIC/DIRECTION  keys  would  be  more  efficient  if 
they  were  arranged  in  the  same  order  as  on  a  ten-key  adding  machine? 

12  3  4 

No  Maybe  Yes  No  Difference 

(Appendix  4  continued  on  following  page)-- 


5.  Which  do  you  feel  would  be  the  best  arrangement  for  the  FLIGHT  ID  keys? 


IN  ORDER  ALTERNATED 

17  1  2 

2  8  3  4 

3  9  5  6 

4  A  7  8 

5  B  9  A 

6  C  B  C 


6.  Did  you  use  one  hand  (which  one)  or  both? 


LEFT 

RIGHT 

BOTH 

none 

none 

none 

25% 

25% 

25% 

50% 

50% 

50% 

75% 

75% 

75% 

100% 

100% 

100% 

7.  Do  you  think  it  would  be  helpful  if  the  ACTION  keys  were  positioned  on  the 
right  side  and  the  ALPHA  keys  were  positioned  on  the  left  side? 

12  3  4 

Not  Helpful  Helpful  Very  Helpful  No  Difference 

8.  Do  you  think  the  position  of  the  ENTER  key  is  best  where  it  is? 

1  2  3 

Better  Somewhere  Else  (Where?)  No  Difference  Best  There 

9.  Do  you  think  the  FUNCTION  keys  are  better  at  the  bottom  or  the  top  of  the 
keyboard? 

1  2  3 

Bottom  Top  Better  Somewhere  Else  (Where?) 

10.  Was  it  helpful  having  two  SPACE  keys? 

12  3  4 

Not  Helpful  Helpful  Very  Helpful  No  Difference 

11.  Did  the  non-functional  keys: 

No  Sometimes  Yes  No  Difference 


Cause  you  to  make  mistakes?  123  4 

Slow  you  down?  123  4 

(Appendix  4  continued  on  following  page)— 
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12.  Do  you  think  that  the  CARRIAGE  RETURN  key  should  be  moved? 

1  2  3 

No  Yes  No  Difference 

13.  Would  it  be  helpful  to  have  the  ACTION  keys  further  apart? 

12  3  4 

Not  Helpful  Helpful  Very  Helpful  No  Difference 

14.  Is  it  confusing  having  the  MIN  ("),  MINI,  and  MAX  ACTION  keys  right  next 

to  each  other? 

12  3  4 

No  Sometimes  Yes  No  Difference 

15.  Were  the  ENTRY  keys  helpful  in  correcting  your  mistakes? 

•  1  2  3  4 

Not  Helpful  Helpful  Very  Helpful  No  Difference 

16.  Which  of  the  following  keys  did  you  use  to  correct  mistakes? 


Never 

Rarely 

Sometimes 

Mostly 

Always 

Delete  Character 

1 

2 

3 

4 

5 

Delete  Line 

1 

2 

3 

4 

5 

Clear  Display 

1 

2 

3 

4 

5 

Would  it  be  helpful 

to  have 

the  DIR 

ACTION  key  closer  to  the 

NUMERIC/ 

DIRECTION  keys? 

12  3  4 

Not  Helpful  Helpful  Very  Helpful  No  Difference 

18.  What  was  your  impression  as  to  the  way  the  keyboard  was  broken  up  into 
FLIGHT  ID,  FUNCTION,  ACTION,  DISPLAY,  ALPHA,  ENTRY,  and  NUMERIC/ 
DIRECTION  segments?  Could  it  be  done  better?  How? 


19.  How  could  the  keyboard  be  made  more  efficient? 
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Appendix  5 

PROPOSED  PILOT  TRAINING  MODEL 


Introduction. 

The  cost  of  pilot  training  involves  both  human  resources  required  to  do 
training  and  use  of  computer  equipment.  To  minimize  the  cost,  both  computer 
equipment  and  training  personnel  must  be  effectively  used.  The  subsequent 
description  outlines  a  model  that  can  be  used  for  pilot  training,  hopefully 
without  a  major  reconfiguring  of  the  hardware  in  the  present  system.  Perhaps 
all  that  is  needed  is  software  stored  on  an  RK06  disk  pack.  The  training  could 
take  place  on  second  or  third  shift  since  equipment  will  be  available.  The 
model  would  be  applicable  and  appropriate  with  either  professional  pilots  or 
Academy  students. 

Model . 

Training  programs  are  systematic  procedures  for  helping  people  learn 
skills,  rules,  or  concepts  that  they  can  apply  to  their  job  situation. 

Learning  is  usually  defined  as  a  relatively  permanent  change  in  thought  or 
action  that  results  from  practice  or  experience. 

The  principles  we  are  most  concerned  with  in  the  pilot  trainee  situation 
have  to  do  with  feedback  and  reinforcement.  Feedback  of  knowledge  to  the 
trainee  on  the  results  of  his/her  behavior  has  strong  reinforcement  properties. 
Most  people  find  "rewarding"  the  knowledge  that  they  have  done  well  and 
"negatively  reinforcing"  the  knowledge  that  they  have  done  poorly.  This  should 
serve  to  promote  learning  of  behavior  that  was  judged  "good"  and  extinction  of 
that  judged  "poor."  If  a  person  is  to  learn  to  respond  correctly  he/she  must 
have  some  way  of  knowing  what  is  and  what  isn't  within  the  acceptable  limits. 
Feedback  gives  this  knowledge. 

Negative  reinforcement  may  have  some  useful  purpose  in  training.  On  the 
other  hand,  it  is  doubtful  that  it  can  serve  usefully  as  the  only  means  of 
shaping  behavior.  If  negative  reinforcement  is  used,  it  should  be  used  in 
conjunction  with  opportunities  for  positive  reinforcement.  That  is,  if  an 
incorrect  response  is  criticized,  it  should  be  done  in  a  fashion  that 
encourages  the  trainee  to  adopt  a  more  appropriate  mode  of  behavior  for  which 
he/she  can  then  be  rewarded. 

Timing  of  feedback  is  of  extreme  importance.  This  is  mainly  because  delays 
increase  the  chances  that  some  other  response  will  be  reinforced,  and  the 
trainee  will  lose  direction  and  become  confused.  The  last  thing  he/she  does 
is  the  most  likely  to  be  reinforced  and  remembered  without  regard  to  whether  it 
be  correct  or  incorrect.  Conditioning  occurs  most  rapidly  when  the  to-be- 
learned  response  is  followed  immediately  by  feedback  on  the  correctness  of  the 
response.  This  is  termed  shaping.  Shaping  is  similar  to  a  guidance  system 
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that  continually  adjusts  or  steers  the  learner  toward  the  desired  behavior 
and  consequently  must  occur  immediately  following  the  behavior. 


In  general  this  principle  says  that  knowledge  of  results  is  a  good  thing, 
and  that  the  sooner  it  comes  after  the  learner's  response,  the  better.  So  it 
would  be  most  beneficial  if  the  pilot  training  system  were  designed  to  provide 
the  pilot  trainee  with  immediate  rewarding  feedback  when  he/she  makes  a 
correct  response  and  feedback  that  will  shape/direct  the  trainee  toward  the 
desired  goal  when  he/she  makes  an  incorrect  response.  The  opportunity  should 
be  given  to  correct  any  error  immediately  after  it  is  made,  since  the  last 
response  made  will  be  the  one  most  likely  to  be  remembered  and  the  one  toward 
which  direct  subsequent  behavior  will  be  directed.  A  pilot  training  system 
of  this  sort  is  illustrated  by  the  flow  chart  shown  at  the  end  of  this 
appendix. 

Explanation  of  Flow  Chart. 

The  parameters  are  specified  off-line  in  building  the  operating  program. 
Parameters  are  the  characteristics  or  values  used  to  determine  an  event.  The 
stimulus  input  file  is  an  independent  data  file  that  contains  the  list  of 
recorded  messages  the  pilot  trainee  will  hear  and  be  required  to  enter  on  the 
CRT.  The  stimulus  input  files  are  built  independently  and  are  coded  or  named. 
First,  the  stimulus  input  file  code  or  name  is  input  as  a  parameter  to  the 
operating  program.  Second,  a  parameter  is  required  in  the  operating  program 
to  establish  a  latency  period.  A  latency  period  is  the  amount  of  time  allowed 
for  the  pilot  trainee  to  respond  to  the  recorded  message  he/she  receives. 
Third,  the  maximum  number  of  incorrect  responses  allowable  before  an  error  is 
recorded  must  also  be  established  as  a  parameter  in  the  operating  program. 

The  fourth  and  last  parameter  to  be  specified  is  the  number  of  sequentially 
correct  re<.  onses  the  student  must  make  before  he/she  is  considered  to  be 
proficient  in  operating  the  pilot  position. 

Once  the  parameters  have  been  specified,  the  operating  program  task  file 
is  built  and  will  run  in  the  manner  shown  on  the  flow  chart.  The  flow  chart 
demonstrates  schematically  the  steps  or  general  flow  of  the  program. 

When  the  program  starts,  the  pilot  trainee  will  receive  the  stimulus,  a 
prerecorded  message.  (This  list  of  prerecorded  messages  is  exactly  matched 
with  the  list  of  messages  in  the  stimulus  input  file  that  was  selected  as  a 
parameter.)  If  possible,  the  audio  system  should  coordinate  with  the  messages 
in  the  stimulus  input  file  through  the  microprocessor.  This  would  lessen 
problems  with  the  synchronization  of  the  prerecorded  messages  and  the  stimulus 
input  file.  A  cassette  should  be  used  for  these  recordings  for  convenience  in 
handling  and  storage.  Another  option  is  individually  operated  cassette 
players,  since  some  pilot  trainees  will  be  able  to  enter  the  messages  more 
accurately  than  others,  and  some  will  make  errors  that  would  slow  everyone 
using  the  same  recording.  These  cassette  players  should  be  modified  so  the 
stimulus  messages  can  be  heard  through  the  headsets.  Also,  it  would  be 
helpful  if  the  cassette  players  had  both  hand  and  foot  controls  for  starting 


or  stopping  the  tape  and  moving  the  tape  forward  or  backward.  The  foot 
control  would  allow  the  hands  to  remain  free  to  enter  the  messages  quickly. 
However,  hand  control  will  be  necessary  for  the  handicapped  who  are  unable  to 
use  a  foot  control.  The  trainee  must  respond  to  the  recorded  message  by 
entering  the  message  on  the  CRT  In  the  proper  format  and  using  the  correct 
symbols  and  characters.  The  operating  program  then  performs  a  "compare"  to 
determine  If  the  response  was  correct  or  Incorrect.  If  the  response  was 
correct, there  will  be  a  reward  In  the  form  of  a  message  displayed  on  the  CRT, 
’’Your  response  was  correct." 

Following  the  reward,  the  sequential  correct  response  counter  will  be 
incremented.  In  the  next  step  the  operating  program  performs  another  "compare'1 
between  the  sequential  correct  response  counter  and  the  proficiency  parameter 
to  determine  If  the  proficiency  requirements  have  been  met.  If  the  trainee 
has  achieved  proficiency,  a  message  will  be  displayed  on  the  CRT,  "You  have 
met  the  proficiency  requirements."  Then,  a  listing  of  the  errors  the  trainee 
incurred  prior  to  reaching  proficiency  and  a  diagnostic  summary  will  be 
printed  on  the  printer.  At  this  point  the  program  terminates. 

If  the  trainee  makes  a  correct  response  but  the  proficiency  requirements 
have  not  been  met,  there  will  be  a  check  for  the  end-of-file.  If  it  is  not 
the  end-of-file,  the  latency  timer  will  be  reset  to  zero  and  there  will  be 
another  stimulus  prompt.  If  it  is  the  end-of-file,  the  error  listing  and 
diagnostic  summary  will  be  printed  on  the  printer  and  the  program  will  stop. 

When  a  pilot  trainee  receives  a  stimulus  and  his/her  response  is  incorrect, 
the  program  will  go  to  an  error  condition.  The  operating  program  will  deter¬ 
mine  the  type  of  error  and  where  the  error  occurred  in  the  input.  The  message 
displayed  on  the  CRT  will  be  "Try  again."  If,  on  the  second  try,  the  response 
is  correct,  the  reward  message  will  follow  and  the  program  will  proceed  as 
previously  described  when  no  error  was  made.  If  the  response  continues  to  be 
incorrect,  the  "Try  again"  message  will  continue  to  be  displayed  until  the 
maximum  number  of  allowable  incorrect  responses  set  as  a  parameter  has  been 
met.  Once  the  maximum  number  of  errors  has  been  reached  or  the  latency  period 
times  out,  the  exact  error  will  be  recorded  on  the  error  file  and  the 
diagnostic  file,  the  sequential  correct  counter  will  be  set  to  zero,  and  the 
correct  input  message  will  be  displayed  on  the  CRT  with  a  message  telling  the 
trainee  to  enter  the  correct  response  under  the  corrected  message  listed  on  the 
CRT.  If  the  correct  response  is  entered,  a  reward  message  will  follow  and  the 
program  will  continue  the  same  as  when  the  initial  input  was  correct.  If  again 
the  response  is  not  correct,  the  message  displayed  on  the  CRT  will  be  "Call 
the  instructor  for  assistance."  Again,  the  correct  input  message  will  be 
displayed,  and  the  trainee  must  enter  the  correct  message.  This  will  be  a 
continuous  loop  until  the  correct  response  is  entered.  When  the  correct 
response  is  entered,  a  reward  message  will  follow  and  the  program  will 
continue  as  before. 

If  the  pilot  trainee  receives  a  stimulus  and  does  not  respond  within  the 
time  limit  specified  in  the  parameters,  the  program  will  go  directly  to  the 
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error  and  diagnostic  file,  the  sequential  correct  counter  will  be  set  to 
zero,  and  the  program  will  continue  as  described  when  an  incorrect  response 
is  made. 


The  error  listing  output  at  the  termination  of  the  program  is  just  a 
simple  chronological  listing  of  the  exact  errors  made.  The  diagnostic  output 
consists  of  a  summary  or  frequency  distribution  of  the  types  of  errors 
committed,  indicating  the  apparent  types  of  messages  where  remedial  work  is 
needed.  Practice  stimulus  input  files  could  then  be  built  and  used  on  the 
same  operating  program  where  stimulus  input  messages  of  a  certain  type  are 
heavily  emphasized. 


Set  Parameter*  1 

t.  Stimulus  input  file 

2.  Set  latency  period 

3.  Set  maximum  of  incorrect 

4.  Set  proficiency  (•*  #f 

responses  before  error 

sequentially  correct 

i»  recorded 

responses  required) 

C :  To  be  done 

off  Ime 

